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SUMMARY 


This  report  describes  the  development  and  analysis  of 
a SAINT  model  of  a real-time  simulation  of  a Remotely  Piloted 
Vehicle/Drone  Control  Facility  (RPV/DCF) . The  major  component 
of  the  real-time  simulation  is  a mock-up  of  a DCF  in  which 
operators  monitor  and  control  the  flight  of  simulated  RPVs 
through  the  use  of  cathode  ray  tube  (CRT)  displays  of  RPV 
flight  paths  and  parameters. 

In  this  research  effort,  a general  purpose  digital 
computer  was  used  to  model  the  real-time  simulation.  The 
modeling  vehicle  used  was  the  previously  developed  digital 
computer  program  called  SAINT.  SAINT,  Systems  Analysis  of 
Integrated  Networks  of  Tasks,  is  a man-machine  modeling  and 
simulation  technique  through  which  the  transformation  from 
a real-time  to  a digital  simulation  can  be  accomplished. 

SAINT  provides  the  simulation  concepts  necessary  to  model 
systems  that  contain  both  tasks  (discrete  elements)  and 
state  variables  (continuous  elements) . The  SAINT  model 
presented  in  this  report  includes  both  of  these  elements  as 
well  as  interactions  between  the  elements.  The  state  variable 
portion  of  the  model  duplicates  the  simulation  of  RPV  flight 
of  the  real-time  simulation.  The  task-oriented  portion 
represents  the  control  and  decision  tasks  performed  by  the 
DCF  operators.  The  interactions  between  the  elements  include 
the  presentation  of  mission  status  information  to  the  operators 
and  the  processing  of  commands  sent  to  the  RPVs  by  the 
operators . 

The  SAINT  model  of  the  RPV/DCF  is  developed  to  be  appli- 
cable for  all  operator  groups  and  all  missions  performed. 

The  SAINT  model  simulates  a specific  mission  through  the 
insertion  of  input  values  that  describe  the  mission. 

The  validity  of  the  SAINT  model  of  the  RPV/DCF  was 
evaluated  by  comparing  mission  performance  outputs  of  the 
SAINT  and  real-time  simulations.  This  procedure  has  resulted 
in  a SAINT  model  of  the  RPV/DCF  real-time  simulation  that  is 
valid  for  one  team  and  one  mission  and  which  demonstrates 
the  applicability  of  the  SAINT  technique  for  the  study  of 
RPV/DCF  and  other  complex  man-machine  systems. 

This  report  describes  the  RPV/DCF  real-time  simulation, 
the  SAINT  model  of  the  system,  the  model  evaluation  procedures, 
and  the  results  of  the  validation  process.  The  results  are 
presented  in  tabular  and  histogram  form.  In  addition,  this 
report  outlines  procedures  for  using  the  SAINT  model  in 


conjunction  with  the  real-time  simulation  to  efficiently 
analyze  RPV/DCF  system  design.  Procedures  for  expanding 
the  SAINT  model  are  also  described. 
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SECTION  I 


INTRODUCTION 


During  the  past  several  years,  a tool  has  been  developed 
to  assist  in  the  design  and  analysis  of  complex  man-machine 
systems  by  providing  a general  framework  within  which  a wide 
variety  of  systems  can  be  modeled.  SAINT,  Systems  Analysis 
of  Integrated  Networks  of  Tasks,  provides  the  simulation 
concepts  necessary  to  model  the  man  and  the  machine  in  the 
face  of  environmental  factors  { 1 , 2 , 3 , 4 , 5 , 6 ) . 


Independent  of  this  effort  has  been  the  development  of 
a real-time  computer  simulation  of  a multi-operator  Remotely 
Piloted  Vehicle/Drone  Control  Facility  (RPV/DCF) . This 
real-time  simulation  is  currently  being  used  to  evaluate 
operator  and  system  parameters  relevant  to  RPV  design  and 
performance.  The  major  component  of  the  system  is  a mock-up 
of  a DCF,  where  actual  operators  c ntrol  the  flight  of 
simulated  RPVs  through  the  use  of  CRT  displays  of  RPV  flight 
paths  and  parameters. 

This  report,  along  with  a companion  report  (9),  documents 
the  development  and  analysis  of  a SAINT  computer  simulation 
model  of  the  real-time  RPV/DCF  simulation. 


Objectives  of  the  SAINT  Modelin<  fort 

The  SAINT  model  is  constructed  to  duplicate  the 
RPV/DCF  real-time  simulation  and  to  output  measures  of 
system  performance  that  can  be  used  to  study  RPV  system 
design.  Since  this  is  the  initial  phase  of  the  SAINT 
RPV/DCF  modeling  effort,  the  objectives  of  this  effort  are 
to  demonstrate  the  applicability  of  the  SAINT  modeling 
technique  to  the  analysis  of  the  RPV  system  and  to  indicate 
how  future  efforts  should  be  constructed  and  coordinated 
with  the  real-time  simulation  so  that  an  effective  analysis 
of  RPV  system  design  and  operator  requirements  can  be 
obtained . 

The  SAINT  model  of  the  RPV/DCF  real-time  simulation  is 
the  first  major  application  of  the  revised  SAINT  simulation 
language  (SAINT  III).  Thus,  another  objective  of  this 
effort  is  to  verify  the  operation  of  the  SAINT  simulation 
program.  Also,  since  the  RPV/DCF  ;imulation  is  a highly 
complex  man-machine  system,  the  SA  NT  RPV/DCF  model  will 
establish  SAINT  as  a:  effective  technique  in  the  analysis 

of  complex  systems. 


Scope  of  the  SAINT  Modeling  Effort 
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The  RPV/DCF  real-time  simulation  referred  to  as  "RPV  II" 
is  modeled  in  SAINT  for  this  contract.  The  mission  structure  : 

of  RPV  II  is  explained  in  Section  II  of  this  report.  However, 
since  this  is  the  initial  modeling  effort  of  the  RPV/DCF  simu- 
lation in  SAINT,  the  evaluation  procedure  will  be  performed 
for  only  one  team  and  one  mission  of  the  real-time  simulation 
study.  The  continuation  of  this  effort  beyond  this  contract 
should  include  the  analyses  of  additional  missions,  operator 
teams,  and  system  configurations. 

Report  Structure  j 

This  report  describes  the  development  and  analysis  of 
the  SAINT  model  of  the  RPV/DCF  simulation.  The  technical 
documentation  of  the  SAINT  model  is  contained  in  a companion 
report  (AMRL-TR-75-119) . Section  II  describes  the  real-time 
RPV/DCF  simulation  including  the  computer  simulation  of  RPV 
flight,  the  CRT  console  display  and  keyboard,  and  the 
operators'  responsibilities  and  performance.  The  information 
detailed  in  this  section  was  obtained  from  extensive  dis- 
cussions with  AMRL  personnel  responsible  for  the  facility, 
through  interviews  with  the  operators,  from  observations 
of  the  simulation  in  operation,  through  actual  experience 
at  the  CRT  consoles,  and  from  reference  material  (7,8). 

Section  III  presents  the  SAINT  model  of  the  RPV/DCF 
real-time  simulation  described  in  Section  II. 

Section  IV  defines  the  model  evaluation  procedure.  It 
includes  the  performance  variables  considered,  the  validation 
procedures  applied,  and  the  results  of  the  analysis. 

Section  V presents  the  conclusions,  and  Section  VI  makes 
recommendations  concerning  the  use  and  embellishment  of  the 
SAINT  model  of  the  RPV/DCF  simulation. 


SECTION  II 


THE  RPV/DCF  REAL  TIME  SIMULATION 


The  Remotely  Piloted  Vehicle/Drone  Control  Facility 
(RPV/DCF)  system  model  developed  at  the  Aerospace  Medical 
Research  Laboratory  ( AMRL ) is  designed  to  simulate,  in  a 
real-time  environment,  a mission  consisting  of  multiple 
groups  of  RPVs  flying  to  a target  and  returning  to  home 
base  (7,8).  RPVs  are  launched  on  a mission  in  groups  of 
three:  a strike  (S)  RPV  carrying  payload,  an  electronic 

(E)  RPV  carrying  jamming  equipment,  and  a reconnaissance 
(L)  RPV  carrying  film  recording  equipment.  The  flight  of 
these  RPVs  is  coordinated  so  that  the  S and  E RPVs  arrive 
at  the  target  together  followed  by  the  L RPV.  The  E RPV 
causes  electronic  disturbances  in  the  enemy's  defense 
communications,  weapon  control  mechanisms,  or  radar  sensors 
to  protect  the  S RPV,  while  the  L RPV  photographs  the  target 
after  the  strike  for  purposes  of  damage  assessment.  Up  to 
eleven  groups  of  RPVs  may  take  part  in  a single  mission, 
with  an  additional  L RPV  following  the  groups  taking  wide 
area  photographs  of  the  entire  target  area. 

The  real-time  RPV/DCF  simulation  consists  of  two 
components.  The  first  is  a computer  simulation  of  RPV 
flight  which  monitors  and  computes  the  status  of  each  RPV 
during  the  entire  mission.  In  this  simulation,  RPV  flight 
is  affected  by  errors  in  onboard  navigation  and  position 
reporting  systems,  by  enemy  electronic  interference  and 
defenses,  and  by  equipment  malfunctions.  Due  to  these 
considerations,  RPVs  may  be  lost  or  may  fly  off  course,  and 
thus  require  external  monitoring  and  control.  The  second 
component  of  the  real-time  system,  the  DCF  operators,  provide 
that  control. 

The  digital  simulation  program  updates  the  status  of 
each  RPV  every  five  seconds  and  supplies  this  information 
to  the  operators  through  CRT  displays.  This  status 
information  consists  of  estimated  times  of  arrival  (ETAs) , 
flight  paths,  velocities,  altitudes,  fuel  levels,  target 
coordinates,  etc.  The  DCF  operators  utilize  this  information 
to  control  the  progress  of  the  mission.  To  correct  the 
RPV  flight  errors  and  to  perform  their  other  mission  duties, 
the  DCF  operators  send  commands  via  the  CRT  consoles  to 
the  simulated  RPVs.  A light  pen  and  CRT  terminal  keyboard 
are  used  to  generate  the  commands.  Thus,  RPV  flight  is 
simulated  by  a digital  computer  which  accepts  changes  in 
RPV  flight  parameters  that  are  input  to  the  CRT  consoles 
by  the  operators. 
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Simulation  of  RPV  Flight 

At  launch,  each  RPV  is  assigned  a flight  path  which  is 
assumed  to  be  optimal  in  terms  of  terrain  and  defense 
avoidance.  This  flight  path  is  an  input  to  the  mission.  The 
geographic  area  over  which  the  RPVs  fly  and  two  typical  paths 
are  shown  in  Figure  1.  Each  flight  path  is  divided  into 
three  phases.  The  enroute  phase  includes  the  RPVs  launch 
from  a launch  site  in  the  safe  zone  - that  area  below  the 
Forward  Enemy  Battle  Area  (FEBA)  line  - and  the  flight  to  the 
target  area;  the  terminal  phase  involves  RPV  flight  within 
the  target  area;  and  the  return  phase  refers  to  the  flight 
of  the  RPV  from  the  target  area  to  the  recovery  area  in  the 
safe  zone.  The  flight  paths  for  the  RPVs  are  stored  onboard 
each  RPV  in  computers  that  control  RPV  flight  during  the 
enroute  and  return  phases.  During  the  terminal  phase,  S RPVs 
are  controlled  by  a pilot  at  the  DCF  via  a TV  camera  in  the 
nose  of  the  RPV.  The  real-time  simulation  employs  a tele- 
vision camera  - terrain  board  installation  during  this  phase 
of  terminal  flight.  RPVs  that  are  not  flown  in  this  manner 
(E  and  L RPVs)  through  the  terminal  phase  are  flown  by 
pseudo-pilots  which  are  controlled  by  the  RPV  flight  simulation 
program  and  fly  the  RPVs  according  to  onboard  flight  path 
instructions . 


In  addition  to  a flight  path,  each  RPV  has  a flight 
profile  that  indicates  the  altitudes  and  velocities  that  the 
RPV  should  fly  during  different  phases  of  the  mission.  All 
RPVs  are  designed  to  fly  at  an  altitude  of  200  feet  during 
enroute,  to  "pop-up"  to  3000  feet  for  the  terminal  phase, 
and  then  to  "pop-down"  to  200  feet  during  return.  The  alti- 
tude of  the  RPVs  for  enroute  and  return  is  set  low  (200  feet) 
to  avoid  detection  and  destruction  by  enemy  defenses.  The 
velocity  profiles  for  the  RPVs  depend  on  RPV  type.  E and  L 
RPVs  are  designed  to  fly  the  entire  mission  at  400  knots. 

S RPVs  are  designed  to  fly  the  enroute  and  return  legs  at 
400  knots  and  the  terminal  phase  of  the  mission  at  250  knots. 
However,  the  three  types  of  RPVs  should  be  coordinated  so 
that  they  reach  the  target  area  together,  while  recoveries 
should  be  separated  in  time.  Thus,  changes  are  made  to  RPV 
velocity  so  that  the  RPV  reaches  the  target  and  recovery 
areas  at  required  times.  For  purposes  of  the  real-time 
simulation,  velocity  and  altitude  changes  are  considered 
instantaneous.  Navigation  system  errors  affecting  altitude 
are  assumed  negligible. 
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RPV  Characteristics 


Since  the  real-time  simulation  is  designed  to  study  the 
RPV  system,  and  since  the  exact  specifications  of  RPVs  to 
be  used  in  a real-world  RPV  system  are  presently  unknown, 
the  real-time  system  is  constructed  based  on  assumptions 
concerning  RPV  design  and  performance  which  appear  to  be 
the  best  representation  of  the  future  working  system. 


The  maximum  speed  of  an  RPV  is  475  knots;  the  minimum 
speed  is  250  knots.  Outside  this  range,  the  RPV  will  crash. 

Each  RPV  is  given  a fixed  amount  of  fuel  at  the  time  of 
launch;  as  specified  on  mission  input.  The  fuel  consumption 
rate  of  the  RPVs  is  dependent  on  RPV  velocity  and  altitude 
and  is  determined  from  a fuel  consumption  table  read  on 
mission  input.  The  RPVs  have  no  altitude  restrictions.  How- 
ever, at  low  altitudes,  terrain  avoidance  becomes  a consideration; 
while  at  high  altitudes,  defense  avoidance  must  be  considered. 


The  turning  capability  or  maneuverability  of  an  RPV 
is  completely  defined  given  a specific  set  of  RPV  charac- 
teristics (wingspan,  weight,  length,  thrust,  etc.).  In 
most  cases,  this  capability  is  put  in  terms  of  "g-load". 
The  "g-load"  capability  is  defined  in  terms  of  minimum 
possible  radius  of  turn  and  is  set  at  one  nautical  mile. 
All  RPV  turns  are  made  at  the  minimum  possible  radius  of 
turn . 

Navigation  Systems 


A real-world  RPV  will  attempt  to  achieve  its  flight 
path  by  adjusting  its  flight  control  surfaces  and  speed  in 
response  to  momentary  changes  in  RPV  position.  These 
changes  may  be  due  to  wind  currents,  turn  execution,  etc. 
However,  the  RPV  requires  a mechanism  to  detect  these  changes 
and  this  mechanism  is  the  navigation  system.  If  the  RPVs 
were  given  perfect  navigation  systems,  no  RPV  would  fly 
off  its  flight  path  and  there  would  be  no  errors.  Unfortu- 
nately, no  navigation  system  is  perfect  and  errors  occur. 
These  errors  are  manifested  in  three  principal  ways  relative 
to  the  flight  path: 

1)  lateral  deviation  or  ground  course  deviation, 
which  represents  the  perpendicular  distance 
from  the  desired  flight  path; 


2) 


along  track  or  ground  speed  deviation,  which 
represents  the  distance  ahead  or  behind  the 
desired  flight  path  position  that  the  true 
position  lies;  and 


i 


3)  expected  time  of  arrival  (ETA)  deviation,  which 
represents  the  time  differential  between  the 
flight  path  ETA  and  the  actual  times  of  arrival 
at  some  specific  geographic  point. 

These  errors  are  illustrated  in  Figure  2. 

There  are  many  types  of  navigation  systems,  each  with 
a different  expected  accuracy.  The  real-time  simulation 
considers  only  dead  reckoning  navigation  systems;  navigation 
systems  that  cannot  measure  position.  They  have  knowledge 
of  only  two  quantities:  the  time  of  day  and  an  estimated 
ground  speed  vector.  This  vector  is  composed  of  a ground 
course  (GC)  estimate  and  a ground  speed  (GS)  estimate. 

An  RPV  with  one  of  these  navigation  systems  will  assume 
that  the  estimated  ground  speed  vector  is  accurate  (which 
it  will  not  be)  and  act  accordingly,  thus  causing  flight 
error.  This  error  will  accumulate  if  not  corrected  by 
an  outside  force,  since  the  RPV  will  not  realize  that  there 
is  any  error  at  all.  Three  types  of  dead  reckoning  systems 
are  simulated:  basic,  doppler,  and  inertial.  Each  RPV 

starts  the  mission  with  the  inertial  navigation  system 
operative  and  the  basic  and  doppler  systems  as  back-up. 

The  errors  of  the  GC  and  GS  estimates  made  by  the  navigation 
systems  are  normally  distributed  and  the  parameters  of 
these  errors  for  each  RPV  are  read  on  mission  input. 

Communication  Links 

For  the  DCF  to  be  aware  of  RPV  status  and  for  the  RPV 
to  receive  commands  from  the  DCF,  there  must  be  communication 
between  the  RPV  and  DCF.  Since  communication  links  operate 
on  a straight  line  basis,  a relay  aircraft  must  be  used  to 
overcome  the  interference  effects  of  high  terrain  and  the 
curvature  of  the  earth.  The  three  data  links  modeled  in 
this  simulation  are  illustrated  in  Figure  3.  The  command 
data  link  (CL)  is  used  to  transmit  instructions  to  the  RPV 
from  the  DCF  and  to  confirm  the  reception  of  these  instruc- 
tions to  the  DCF  from  the  RPV.  The  TV  video  data  link 
(TV)  is  used  to  transmit  television  pictures  from  the  nose 
of  the  RPV  when  it  is  over  the  target  area.  It  is  used 
only  over  the  target  area  so  that  the  RPV  is  as  radio  silent 
as  possible  during  the  other  mission  phases.  The  position 
reporting  data  link  (PR)  transmits  radio  signals  from  the 
RPV  that  are  converted  into  RPV  position  reports  at  the 
DCF.  Due  to  interference  and  imperfections  in  communications 
equipment,  PR  reports  are  never  exact. 

Position  report  errors  arise  from  two  conditions.  First, 
the  time  required  for  a signal  from  the  relay  aircraft  to 
reach  the  RPV  and  return  to  the  relay  aircraft  is  measured. 
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Figure  2.  Navigation  System  Errors. 
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Figure  3.  Pictoral  Description  of  RPV  Data  Links 


The  time  value  observed  is  used  to  estimate  the  distance 
between  the  RPV  and  the  relay  aircraft.  This  distance 
estimate  has  an  error  that  is  independent  of  the  distance 
between  the  RPV  and  the  relay  aircraft. 

The  second  condition  causing  PR  error  is  directional 
error.  In  receiving  a signal  from  the  RPV,  the  relay 
aircraft  provides  an  estimate  of  the  direction  of  the  RPV 
from  the  relay.  This  quantity  also  contains  errors. 

These  errors  become  more  serious  as  the  distance  between 
the  RPV  and  the  relay  aircraft  increases.  For  example,  a 
1 degree  error  at  a distance  of  57  miles  will  produce  a 
measurement  error  of  1 mile,  while  the  same  angular  error 
at  114  miles  will  produce  a two  mile  error. 

The  DCF  obtains  estimates  of  the  direction  and  distance 
of  the  RPV  from  the  relay  aircraft  and  converts  these  to 
an  estimate  of  the  position  of  the  RPV.  This  is  the  PR 
position,  and  contains  both  the  distance  and  direction 
errors.  A new  random  PR  error  is  incurred  for  every  position 
report.  The  parameters  for  the  normally  distributed 
position  report  range  and  angular  errors  are  read  on 
mission  input. 

In  addition  to  the  errors  they  impose,  communication 
links  are  subject  to  failure.  For  the  real-time  simulation, 
there  are  two  causes  of  data  link  failure:  malfunctions 
and  jamming.  When  a data  link  malfunctions,  it  remains 
inoperative  for  the  entire  mission.  All  three  types  of 
data  links  are  subject  to  malfunctions.  Jamming,  on  the 
other  hand,  will  cause  only  a temporary  data  link  outage. 

Jamming  is  a condition  that  occurs  only  around  the 
target  area.  It  is  caused  by  radio  stations  set  up  by  the 
enemy  to  broadcast  interference  on  the  same  frequency  of 
the  relay  to  RPV  broadcasts.  If  the  enemy  is  successful, 
the  communication  links  between  the  relay  and  RPV  are 
broken  and  no  valid  information  can  be  sent.  It  is  assumed 
that  the  enemy  has  placed  a radio  station  or  "jammer" 
every  2.5  miles  across  the  front  of  the  target  area  on  a 
line  165  miles  from  the  FEBA  line,  as  illustrated  in 
Figure  4.  If  the  RPV  is  flying  in  the  shaded  area  of 
Figure  4,  it  has  a chance  of  being  jammed.  In  the  white 
areas,  the  messages  are  transmitted  successfully.  (Note 
that  RPV  status  is  monitored  every  five  seconds;  at  each 
five  second  interval,  the  positon  of  the  RPV  will  be  used 
with  the  probability  of  message  error  at  that  position 
to  determine  if  the  RPV  is  jammed.  If  it  is,  it  will  be 
jammed  for  five  seconds.)  The  probability  of  an  RPV  being 
jammed  when  it  is  in  the  shaded  area  depends  upon  its 
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Y-coordinate . A table  read  on  mission  input  provides 
the  probability  values  with  respect  to  the  Y-coordinate 
values . 

When  data  links  become  inoperative,  the  operation 
of  the  DCF  and  its  operators  are  affected.  If  TV  is 
down,  it  cannot  be  used.  There  are  no  replacements  or 
possible  repairs  for  TV.  If  it  is  down  for  an  RPV,  the 
terminal  pilot  will  not  fly  the  RPV  through  the  target 
area.  If  CL  goes  down  for  an  RPV,  commands  cannot  be 
sent  to  the  RPV  and  it  will  continue  flying  on  stored 
instructions.  If  the  malfunction  is  permanent,  the 
RPV  is  usually  dropped  from  the  system,  as  it  can  no 
longer  be  controlled. 

In  other  cases,  £he  CL  outage  will  be  temporary  due 
to  jamming.  At  the  end  of  any  period  of  five  seconds, 
the  DCF  attempts  to  send  commands  to  the  RPVs . If  CL  is 
down  for  an  RPV,  the  command  is  saved  for  five  seconds 
and  then  resent.  However,  if  new  commands  are  made  for 
the  same  RPV  while  CL  is  down,  the  old  commands  of  the 
same  type  will  be  lost  and  only  the  new  commands  sent. 

Since  the  operators  need  to  know  the  positions  of 
the  RPVs,  the  DCF  computer  will  estimate  the  position 
of  the  RPV  whenever  the  PR  data  link  is  down  for  an 
RPV.  It  will  do  this  by  using  the  last  PR  point  and 
computing  the  RPV  position  as  if  it  came  from  that  PR 
point  and  perfectly  executed  the  onboard  flight  commands 
during  the  last  frame.  This  PR  estimate  will  contain 
errors  because  1)  the  last  PR  point  was  itself  in  error, 
and  2)  the  RPV  will  not  follow  the  flight  path  commands 
perfectly.  Naturally,  the  longer  the  PR  link  is  down, 
the  more  the  PR  prediction  is  degraded.  However,  this 
extrapolation  is  the  best  position  estimate  available 
and  operators  must  act  using  this  estimate. 

RPV  Reliability  and  Vulnerability 


RPVs  are  not  infallible.  In  fact,  since  RPVs  are 
designed  to  be  inexpensive  and  pilot-free  with  little 
redundanc  ■'  in  design,  failures  are  to  be  expected.  RPV 
failures  during  a mission  and  the  times  of  these  failures 
are  determined  from  mission  input.  The  types  of  failures 
and  their  consequences  to  the  RPVs  are  given  in  Table  I. 
Note  that  failure  types  1,  2,  3,  4,  5,  and  6 result  in 
RPVs  leaving  the  system.  These  will  be  referred  to  as 
"terminal"  malfunctions. 

RPV  survival  depends  not  only  on  reliability,  but 
also  on  altitude,  deviation  from  flight  path,  and  the 


TABLE  I 


RFV  FAILURES  AND  CONSEQUENCES 


FAILURE  TYPE 


CONSEQUENCE  TO  RPV 


Roll 


Thrust 


6.  Onboard  Computer 


RPV  out  of  mission 


7.  Current  Navigation  System 


RPV  will  fly  off  course 
until  a new  navigation 
system  is  put  in  operation 
by  DCF 


Communication  Links 


Links  down  for  rest 
of  mission 


resources  of  the  enemy.  Taking  all  these  factors  into 
consideration,  the  probability  of  survival  of  an  RPV 
has  been  made  a function  of  altitude  and  lateral  deviation 
for  the  simulation.  These  probabilities  are  given  in  terms 
of  probability  of  survival  for  five  seconds.  Thus, 
every  five  seconds,  the  position  of  the  RPV  will  be 
recorded  and  the  survival  probability  of  this  RPV  will  be 
tested.  A table  of  survival  probabilities  based  on 
altitude  and  lateral  deviation  is  given  on  mission  input. 

Operator  Activities 

There  are  five  persons  stationed  in  the  mock-up  of 
the  DCF.  Four  of  them  act  as  enroute/return  operators  and 
are  seated  at  the  CRT  consoles.  The  fifth  person  acts  as 
the  terminal  pilot,  whose  only  responsibility  is  to  fly  S 
RPVs  through  the  terminal  phase  of  the  mission  using  the 
TV  camera-terrain  board  installation. 

The  information  concerning  operator  activities  presented 
below  was  obtained  from  extensive  discussions  with  AMRL 
personnel  responsible  for  the  facility,  through  interviews 
with  the  operators,  from  observations  of  the  simulation 
in  operation,  through  actual  experience  at  the  CRT  consoles, 
and  from  reference  material  (7,  8). 

CRT  Consoles  and  Functional  Keyboards 


DCF  operators  are  informed  of  mission  status  through 
the  CRT  terminals  in  front  of  them.  In  addition,  operators 
send  commands  to  the  RPVs  and  instructions  to  the  terminals 
through  a function  keyboard.  The  CRT  display,  as  pictured 
in  Figure  5,  contains  five  separate  information  blocks: 

1)  Geographical  Display:  This  area  displays  a two 

dimensional  view  of  RPV  flight.  The  information 
that  can  be  displayed  for  any  or  all  RPVs  includes 
flight  paths,  major  waypoints,  track  signatures 
(reports  of  the  position  of  the  RPVs) , and  targets. 
This  display  has  three  scales  (MOD  1,  MOD  2,  and 
MOD  3)  which  control  the  size  of  the  display. 

MOD  1 makes  the  display  200  x 200  nm  (nautical 
miles) . This  places  the  entire  geographical 
area  of  the  simulation  in  view.  MOD  2 and  MOD  3 
cause  the  display  to  be  20  x 20  nm  and  7x7  nm, 
respectively,  centered  about  the  position  of  the 
individual  RPV  that  the  operator  is  viewing. 

Thus,  by  controlling  the  geographical  display, 
the  DCF  operator  can  monitor  the  entire  mission  or 
concentrate  on  the  position  of  a single  RPV. 
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Figure  5.  Sample  CRT  Display. 


2) 


Menu  Area:  This  area  displays  information  about 

the  status  of  all  RPVs.  It  gives  the  RPV  number, 
the  next  major  waypoint  of  the  RPV  and  the  ETA  to 
that  point,  the  command  link  status,  the  flight 
mode  indicator,  and  the  lateral  deviation  number. 

The  major  waypoints  for  RPVs  are  designated  by 
S,  H,  B,  and  R.  Operators  are  required  to 
perform  special  actions  when  an  RPV  reaches  those 
points  in  its  flight.  The  command  link  status 
indicator  tells  the  operator  whether  or  not 
commands  can  be  sent  to  the  RPV  at  this  time. 

The  flight  mode  indicator  tells  the  operator  if  a 
terminal  pilot  is  controlling  the  RPV.  The  lateral 
deviation  number  indicates  the  number  of  consecutive 
frames  (five  second  intervals)  that  the  lateral 
deviation  of  the  RPV  has  been  greater  than  the 
lateral  deviation  alarm  threshold  (specified  on 
mission  input) . Also  appearing  in  the  menu  area 
is  a malfunction  indicator  to  inform  the  operators 
when  a malfunction  occurs. 

3)  Status  Block:  The  status  block  is  accessed  by  the 

operator  if  detailed  status  information  concerning 
one  particular  RPV  is  required.  The  information 
displayed  includes  RPV  number,  RPV  type,  velocity, 
altitude,  fuel  level,  fuel  rate,  and  navigation 
system  in  use. 

4)  Outstanding  Commands:  This  area  will  display  the 

commands  initiated  by  the  operator  for  an  RPV 
that  have  not  yet  been  sent  to  the  RPV.  These 
include  altitude  changes,  velocity  changes, 
navigation  system  changes,  directional  changes, 
requests  to  destruct  the  RPV,  and  requests  to  have 
the  RPV  deploy  chutes. 

5)  Message  Block:  This  area  displays  instructions 

from  the  DCF  computer  to  the  operator  pertaining 
to  the  operation  of  the  console. 

The  information  displayed  on  the  consoles  is  updated  every 
five  seconds,  giving  the  operators  information  concerning 
mission  status  only  at  discrete  five  second  intervals  of 
time . 


Operators  send  commands  to  the  RPVs  and  instructions  to 
the  CRT  terminals  through  the  use  of  the  functional  keyboard, 
the  terminal  keyboard,  and  a light  pen  (LP)  which  recognizes 
positions  on  the  CRT  display.  RPVs  are  selected  from  the 
menu  by  light  penning  the  RPV  number  or  geographical  position 
of  the  RPV  in  the  geographical  display.  The  functional 


keyboard  is  pictured  in  Figure  6.  The  types  of  commands 
the  operators  can  make  and  the  procedures  for  doing  so  are 
given  in  Tables  II,  III,  and  IV. 

Operator  Responsibility 


Each  enroute/return  operator  has  a specific  set  of 
RPVs  under  his  control  during  the  enroute  and  return  phases 
of  the  mission  and  another  set  for  handover  (giving  RPV 
control  to  the  terminal  pilot  or  a pseudo-pilot)  and  handback 
(taking  RPV  control  back  from  the  terminal  pilot  or  a 
pseudo-pilot)  operations. 

RPVs  are  launched  and  fly  in  groups  of  three.  The 
arrival  of  these  RPVs  to  their  respective  hand-off  coordinates 
(the  position  where  the  terminal  pilot  or  pseudo-pilot 
assumes  control) , called  the  H waypoint,  must  be  synchronized 
by  the  operators.  The  E RPV  must  reach  its  H waypoint  15 
seconds  after  the  S RPV  reaches  its  H waypoint,  while  the 
L RPV  reaches  its  H waypoint  2 minutes  after  the  S RPV 
reaches  its  H waypoint.  The  time  allotted  for  each  S RPV 
to  reach  its  H waypoint  i.s  given  to  the  operators  as  the 
mission  begins.  The  ETAs  for  the  associated  E and  L RPVs 
are  computed  from  these  values.  In  addition  to  the  above 
requirements,  S RPVs  are  given  desired  ETAs  to  recovery 
which  operators  must  try  to  satisfy.  The  recovery  of  each 
S RPV  and  the  other  RPVs  must  be  timed  so  that  no  two  RPVs 
are  recovered  within  15  seconds  of  each  other.  Operators 
make  RPV  velocity  changes  in  order  to  meet  the  mission  ETA 
requirements . 

In  addition  to  the  ETA  requirements  discussed  above, 
operators  are  also  required  to  keep  each  RPV  as  close  to 
the  desired  flight  path  as  possible  for  the  duration  of 
the  mission.  This  is  accomplished  by  applying  a directional 
change,  or  patch,  to  the  RPV  when  the  RPV  is  observed  by 
the  operator  as  off  course.  The  decision  to  patch  is  made 
based  on  the  lateral  deviation  number  displayed  in  the  menu 
or  on  the  actual  lateral  deviation  obtained  from  the  RPV 
status  block. 

In  order  to  alter  the  path  of  an  RPV,  the  operator 
indicates,  on  the  geographical  display,  the  points  through 
which  the  RPV  is  to  fly.  The  light  pen  is  used  in  con- 
junction with  a MOD  display  to  generate  the  new  coordinates. 
The  specified  points  will  normally  be  located  between  the 
present  position  of  the  RPV  and  the  original  flight  path. 

The  last  point  specified  is  the  reconnect  point,  and  is 
located  on  the  original  flight  path.  An  example  of  acceptable 
and  unacceptable  patch  points  is  shown  in  Figure  7. 
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TABLE  II 


INSTRUCTIONS 

3UTT0N  PUSHED 
Targets 

General  Times 
Display 

Display  Reset 
Start 


INITIATED  USING  ONLY  THE  FUNCTIONAL  KEYBOARD 


INSTRUCTION 

Display  the  targets  or  reset 
and  do  not  display  the  targets 
if  they  are  already  displayed. 

One  Display  the  whole  geographic 

area  in  its  normal  scale 

(200x200  miles) . 

Blank  the  RPV  status  block. 

Start  the  simulation. 


Cancel  Pending 
Handover 


Cancel  the  handover  just 
requested. 


TABLE  Til 
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INSTRUCTIONS  INITIATED  USING  KEYBOARD  BUTTON 
AND  LIGHT  PENNING  THE  APPROPRIATE  RPV 


BUTTON  PUSHED 


INSTRUCTION 


Turn  on  PR 

Turn  off  PR 

Turn  on  FP 
Turn  off  FP 

Status 
MOD  1 

MOD  2 

MOD  3 


Display  the  position  report 
track  signature  of  the  light 
penned  RPV. 

Turn  off  the  display  of  the 
position  report  track  signature 
of  the  light  penned  RPV. 

Display  the  flight  path  for  the 
light  penned  RPV. 

Turn  off  the  display  of  the 
flight  path  for  the  light  penned 
RPV. 

Display  the  RPV  status  block 
for  the  light  penned  RPV. 

Display  the  normal  200x200  nm  area 
and  prepare  for  patching  to  the 
light  penned  RPV. 

Display  the  20x20  nm  area  around 
the  light  penned  RPV  and  prepare 
for  patching  to  this  RPV. 

Display  the  7x7  nm  area  around 
the  light  penned  RPV  and  prepare 
for  patching  to  this  RPV. 


t 
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TABLE  IV 


COMMANDS  SENT  TO  RPVs  AND  PROCEDURES 
FOR  THEIR  INITIATION 


COMMAND  DESIRED 


Prepare  an  RPV  for 
Handover 


Cancel  the  last 
command  requested 
for  an  RPV 


PROCEDURE  ("PRESS"  REFERS  TO  A 
FUNCTIONAL  KEYBOARD  BUTTON;  "TYPE" 
REFERS  TO  THE  CONSOLE  KEYBOARD) 


Press  "Prepare  for  Handover",  type 
in  the  pilot  or  pseudo-pilot  number 
to  be  handed  to,  type  "EOB" , light 
pen  the  appropriate  RPV. 

Press  "Cancel  Command"  and  light 
pen  the  RPV. 


Change  the  Navigation 
System  of  an  RPV 


Change  the  Altitude 
of  an  RPV 


Change  the  Velocity 
of  an  RPV 


Destruct  an  RPV 
from  the  system 


Press  "Navigation  System" , light 
pen  the  RPV,  type  in  the  new 
navigation  system  code,  type  "EOB". 

Press  "Altitude",  light  pen  the 
RPV,  type  in  the  new  altitude, 
and  type  "EOB". 

Press  "Velocity",  light  pen  the 
RPV,  type  in  the  new  velocity, 
and  type  "EOB". 

Press  "Destruct"  and  light  pen 
the  RPV. 


Drop  an  RPV  from  Press  "Drop"  and  light  pen  the  RPV. 

the  system 


Cause  an  RPV  to  deploy  Press  "Chutes"  and  light  pen 

its  chutes  the  RPV. 


Put  a directional 
patch  on  an  RPV 


Reprogram  an  RPV 


Press  a "MOD"  button,  wait  until  the 
console  display  has  presented  the  MOD 
display  requested,  light  pen  the  points 
the  RPV  is  to  fly  through,  press 
"Reconnect",  and  light  pen  the  point 
where  the  RPV  is  to  be  back  on  the 
flight  path. 

Same  as  above  except  press  "Reprogram 
Reconnect"  instead  of  "Reconnect”. 
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The  patch  point  designated  as  can  be  reached  by 
the  RPV  from  its  present  position  and  heading  by  turning  a 
degrees  to  the  right  at  its  minimum  radius  of  turn  and 
then  flying  straight  to  the  point.  This  is  an  acceptable 
patch  point.  The  patch  point  X2  is  not  acceptable,  as  it 
lies  within  the  minimum  turn  radius  of  the  RPV.  The  patch 
point  X is  also  unacceptable,  as  it  requires  a turn  of 
more  than  90  degrees  (0) . Before  an  operator  can  success- 
fully alter  the  flight  path  of  the  RPV,  he  must  specify 
acceptable  patch  points  to  the  DCF  computer.  If  he  does 
not,  the  patch  will  be  rejected  and  the  RPV  will  continue 
to  fly  on  its  present  course. 

The  number  of  turns  actually  made  by  an  RPV  during 
a patch  is  one  greater  than  the  number  of  patch  points 
specified,  as  a roll-out  turn  is  required  to  achieve  the 
specified  heading  at  the  reconnect  point.  In  addition, 
all  turns  are  made  at  the  minimum  radius  of  turn  to  ensure 
that  the  RPV  flies  the  shortest  possible  distance. 

Another  duty  of  the  operator  involves  the  activities 
in  preparation  for  and  as  a result  of  the  RPV  being  in 
the  target  area.  It  includes  the  pop-up  maneuver,  where 
the  velocity  and  altitude  of  the  RPV  are  changed  for  the 
approach  to  the  target;  the  handoff  activities,  where 
control  of  the  RPV  is  given  to  the  terminal  pilot  or 
pseudo-pilot;  the  handback  activities,  where  control  of 
the  RPV  is  released  by  the  terminal  pilot  or  the  pseudo- 
pilot; and  the  pop-down  maneuver,  where  the  altitude  of 
the  RPV  is  changed  for  the  return  phase.  To  explain  these 
activities,  Figure  8 gives  typical  flight  paths  around  the 
target  area  for  S,  E,  and  L RPVs. 

E and  L RPVs  have  one  major  waypoint  between  launch 
and  target:  H.  Just  prior  to  H,  the  operators  are  required 
to  pop-up  the  RPV  to  3000  feet,  change  the  velocity  to  400 
knots,  prepare  the  RPV  for  handover,  and  signal  which 
pseudo-pilot  will  control  the  RPV  over  target.  At  H,  the 
pseudo-pilot  signaled  (which  is  controlled  by  a console 
operator)  will  flip  the  appropriate  switches  on  his  pseudo- 
pilot control  box  to  accept  control.  The  pseudo-pilot 
will  then  control  the  RPV  through  target  to  the  B (handback) 
waypoint.  Near  B,  control  will  be  released  by  the  pseudo- 
pilot, and  the  console  operator  will  change  the  altitude  of 
the  RPV  to  200  feet  for  the  return  phase. 

For  S RPVs,  there  is  a major  waypoint  before  H called 
S.  It  is  just  prior  to  this  point  that  the  pop-up  maneuver 
is  performed;  the  velocity  is  set  to  250  knots  and  the 
altitude  to  3000  feet.  At  the  S waypoint,  the  RPV  is 
prepared  for  handover  to  the  terminal  pilot.  However, 
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Figure  8.  Typical  Flight  Paths  Around  the  Target  Area 
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control  is  maintained  by  the  console  operator,  who  attempts 
to  patch  between  S and  H to  keep  the  RPV  on  course.  When 
the  RPV  reaches  H,  the  terminal  pilot  takes  control  and 
flies  the  RPV  through  the  target  area  to  B.  Near  B,  he 
releases  control  to  the  console  operator  who  pops-down  the 
RPV  to  200  feet  and  changes  the  velocity  back  to  400  knots. 
Also,  since  the  terminal  pilot  may  have  flown  the  RPV 
widely  off  course,  the  console  operator  may  attempt  one 
patch  to  return  the  RPV  to  its  original  flight  path. 

Since  there  is  only  one  terminal  pilot,  sometimes  two 
S RPVs  are  scheduled  to  reach  the  target  at  approximately 
the  same  time.  When  this  situation  occurs,  one  of  the 
S RPVs  will  be  controlled  by  a pseudo-pilot  during  the 
terminal  phase  instead  of  the  terminal  pilot.  All  other 
operations  on  the  S RPV  will  be  performed  in  the  normal 
manner . 

Aside  from  the  three  general  areas  of  responsibility 
discussed  above,  operators  perform  other  functions  that 
depend  on  mission  conditions.  First,  operators  must  monitor 
the  fuel  supply  to  ensure  that  RPVs  have  sufficient  fuel 
to  return  to  the  recovery  area.  This  activity  is  performed 
when  all  RPVs  have  completed  handoff.  If  a fuel  problem 
is  encountered,  the  velocity  and  altitude  of  the  RPV  are 
changed  to  conserve  fuel.  Second,  operators  must  respond 
to  malfunctions.  If  a terminal  malfunction  occurs,  the 
RPV  must  be  destructed  if  it  is  in  enemy  territory  or  its 
chutes  deployed  if  it  is  in  safe  territory.  If  a CL 
malfunction  occurs,  the  RPV  is  dropped  from  the  simulation. 

If  a navigation  system  malfunction  occurs,  the  navigation 
system  must  be  changed. 

If  an  RPV  is  lost  to  the  system  before  it  reaches  the 
target  area,  the  operator  may  attempt  to  reprogram  the 
mission.  For  example,  an  E RPV  heading  for  recovery  could 
be  used  to  replace  a lost  E RPV  that  had  been  travelling 
to  the  target  area.  The  activities  involved  in  reprogramming 
are:  1)  the  other  two  RPVs  in  the  group  of  three  containing 
the  lost  RPV  are  slowed  to  250  knots,  2)  another  RPV  going  to 
the  same  target  and  of  the  same  type  as  the  lost  RPV  is 
selected  to  replace  it,  3)  as  soon  as  the  replacement  gets 
through  the  target  area,  it  is  rerouted  back  to  the  target 
area,  and  4)  the  ETAs  of  the  replacement  and  the  other 
two  RPVs  are  adjusted  to  make  all  three  arrive  at  the  H 
waypoint  in  the  correct  sequence. 

Operator  Performance 

As  the  simulation  progresses,  the  four  enroute/return 
operators  scan  the  CRT  displays,  request  status  information, 


and  generate  appropriate  commands  to  the  RPVs.  The  sequence 
in  which  operator  actions  are  performed  is  illustrated  in 
Figure  9. 

The  highest  priority  task  for  an  operator  concerns 
the  handover  operation.  From  the  status  information  on 
his  CRT  display,  the  operator  determines  if  it  is  time  to 
prepare  one  of  his  assigned  RPVs  for  handoff  to  the  terminal 
pilot  or  a pseudo-pilot.  If  so,  the  operator  initiates  a 
pop-up  maneuver  on  the  RPV  by  changing  the  altitude  and 
velocity  and  handing  off  the  RPV  to  the  terminal  pilot  or 
pseudo-pilot.  Handoffs  of  RPVs  to  pilots  are  generally 
performed  in  the  following  manner:  S RPVs  controlled  by 
operator  1 to  the  terminal  pilot;  RPVs  controlled  by 
operator  4 to  the  pseudo-pilot  of  operator  1;  E RPVs  to  the 
pseudo-pilot  of  operator  3;  and  L RPVs  to  the  pseudo-pilot 
of  operator  2. 

If  no  handoff  operations  are  to  be  performed,  the  operator 
determines  if  the  terminal  pilot,  or  a pseudo-pilot,  has 
released  control  of  one  of  his  RPVs.  If  so,  he  performs 
a pop-down  maneuver  to  prepare  the  RPV  for  the  return  phase 
of  the  mission.  If  an  operator  has  no  handover  or  handback 
operations  to  perform,  he  checks  the  ETAs  of  his  assigned 
RPVs  which  are  in  the  enroute  or  return  phase  of  the  mission. 

If  there  is  an  ETA  correction  to  be  made  for  an  RPV,  the 
operator  will  change  the  velocity  of  the  RPV.  With  no  ETA 
manipulations  to  perform,  an  operator  will  determine  if 
his  RPVs  are  on  course.  If  not,  he  will  initiate  a direc- 
tional change  or  patch.  The  enroute/return  operator  will 
monitor  mission  status  if  he  has  no  pressing  responsibilities 
(none  of  the  above  actions  was  required) . Once  an  operator 
has  performed  any  action,  he  will  resume  his  activities 
with  his  highest  priority  operation.  Priority  among  RPVs 
for  the  same  operation  is  usually  given  to  the  lower  number 
RPV,  due  to  the  operators'  normal  top-to-bottom  scanning  of 
the  menu. 

Operator  4,  the  "overload  operator",  has  two  additional 
responsibilities  that  are  represented 

in  Figure  9.  First,  prior  to  checking  for  ETA  manipulations, 
he  checks  for  malfunctions  and  compensates  for  them  by 
destructing  an  RPV,  causing  an  RPV  to  deploy  its  parachute, 
or  rerouting  an  RPV  on  the  return  phase  back  to  the  target 
area  to  replace  a lost  RPV.  Second,  he  checks  the  fuel 
levels  on  all  RPVs  during  the  return  phase  of  the  mission. 

If  fuel  conservation  is  necessary,  he  alters  the  velocity 
and  altitude  of  the  RPVs  that  are  low  on  fuel. 

In  observing  the  real-time  simulation  in  operation, 
it  was  found  that  operators  made  infrequent  use  of  a number 
of  keyboard  functions  available  to  them.  "General  Display" 
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Figure  9.  Enroute/Return  Operator  Activity  Sequence 


was  only  used  to  survey  how  close  the  planes  were  to  the 
handoff  point  or  to  check  the  FEBA  line  prior  to  a destruct 
or  deployment  of  chutes  command.  "Display  Reset"  was  never 
used  in  the  simulations  observed.  "Drop"  was  used  only  if 
there  had  been  a CD  malfunction  on  an  RPV.  "Cancel  Handover" 
was  used  only  if  the  handover  command  was  not  accepted  for 
some  unknown  reason  or  if  the  wrong  RPV  number  was  input 
by  the  operator.  Track  signatures  were  generally  called  for 
as  soon  as  the  RPVs  were  launched  and  were  not  turned  off 
until  recovery.  "Cancel  Command"  was  only  used  if  CL  had 
been  down  for  some  time  (commands  could  not  be  cancelled 
fast  enough  under  normal  conditions) . Flight  paths  were  only 
used  for  patches  prior  to  H for  S RPVs,  reprograms,  and  ex- 
tremely large  patches.  Targets  were  usually  displayed 
during  the  entire  mission.  "Start"  was  used  only  at  the 
beginning  of  the  mission.  The  remainder  of  the  keyboard 
functions  were  used  frequently,  as  outlined  above. 

One  primary  responsibility  of  each  operator  is  to  keep 
the  RPVs  on  course.  They  perform  this  activity  by  initiating 
directional  changes,  or  patches,  to  the  RPVs  when  the  RPVs 
have  gone  astray.  In  performing  the  patch,  the  operators 
usually  use  the  MOD  3 display  with  one  patch  point  and  reconnect 
point.  The  patch  point  is  placed  on  the  flight  path  of  the 
RPV  three  dots  above  the  velocity  vector  (the  dots  and  vector 
are  display  elements  that  appear  when  MOD  displays  are 
requested) . The  reconnect  point  is  generally  the  last  dot 
on  the  screen.  However,  the  MOD  2 display  is  used  by  the 
operators  if  the  lateral  deviation  is  as  high  as  3000  feet 
or  if  they  have  been  unsuccessful  in  generating  a patch 
using  a MOD  3 display.  With  a MOD  2 display,  the  patch  point 
is  set  one  or  two  dots  above  the  velocity  vector  and  the 
reconnect  is  two  or  three  dots  above  that.  In  addition, 
patches  are  never  attempted  around  turns  in  the  flight  path. 

Another  of  the  operators'  primary  functions  is  to  set 
up  the  appropriate  ETAs  for  the  RPVs.  This  is  accomplished 
by  making  velocity  changes  for  the  RPVs  and  is  usually  done 
early  in  the  mission  for  ETAs  to  handoff,  and  after  the  RPVs 
have  finished  handoff  for  recovery  ETAs.  To  determine  the 
new  velocity,  a rule  of  thumb  and  experience  is  used.  The 
rule  is:  A one  knot  change  in  the  velocity  will  alter  the 

ETA  by  one-fifth  of  a second  if  the  RPV  is  30  minutes  from 
handoff;  and  the  same  change  will  alter  the  ETA  by  two-fifths 
of  a second  15  minutes  from  handoff.  In  between,  a linear 
approximation  is  used. 

During  the  mission,  the  fuel  supply  of  the  RPVs  must 
be  checked  to  ensure  that  the  RPV  has  sufficient  fuel  to 
reach  the  recovery  area.  Operator  4 checks  these  fuel  supplies 
after  all  RPVs  have  been  through  handoff.  He  determines  if 


the  RPV  will  reach  recovery  at  its  present  fuel  consumption 
rate;  if  not,  he  changes  the  RPV  altitude  and  velocity  in 
order  to  ensure  RPV  arrival  at  the  recovery  area.  It  has 
^ been  determined  by  the  operators  that  a velocity  of  310 

knots  at  an  altitude  of  10,000  feet  is  most  efficient  in  terms 
of  fuel  consumption. 
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SECTION  III 


THE  SAINT  MODEL  OF  THE  RPV/DCF  REAL-TIME  SIMULATION 


The  development  of  a non-real- time  computer  simulation 
model  can  be  of  tremendous  value  when  used  in  conjunction 
with  the  real-time  simulation  for  the  evaluation  of  alternative 
system  configurations  and  operational  procedures.  The 
advantages  from  this  type  of  cooperative  study  are  in 
computer  time,  personnel  organization,  and  expense.  Also, 
as  with  any  major  computer  application,  the  value  of  a 
back-up  system  to  verify  and  check  programming  procedures 
is  incalculable.  Thus,  a model  of  the  real-time  RPV/DCF 
simulation  was  constructed  and  validated  against  the  real- 
time system  using  the  newly  developed  man-machine  simulation 
language,  SAINT.  The  model  development  and  subsequent 
analysis  of  results  demonstrate  the  power  and  usefulness 
of  SAINT  in  the  analysis  of  man-machine  systems.  This  section 
presents  the  SAINT  simulation  of  the  RPV/DCF  real-time 
simulation  currently  being  used  to  study  RPV/DCF  system 
design  and  operation. 
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The  SAINT  model  of  the  real-time  RPV/DCF  simulation 
has  been  built  to  accurately  represent  the  real-time  simulation 
discussed  in  Section  II.  An  overview  of  the  SAINT  model  is 
presented  in  Figure  10.  The  state  variable  model  component 
consists  of  the  simulation  of  RPV  flight  positions,  navigation 
system  errors,  acceptance  of  operator  commands  by  the  RPVs , 

RPV  maneuverability  limitations,  fuel  consumption,  and  the 
effect  of  flight  disturbances.  The  task-oriented  model 
component  includes  all  control  and  decision  tasks  performed 
by  the  DCF  operators  during  a mission.  The  interactions 
between  the  state  variable  and  task-oriented  model  components 
represent  the  display  of  status  information  on  the  operators' 
consoles  and  the  transmission  of  commands  to  the  RPVs. 


State  Variable  Model  Component 

The  state  variable  portion  of  the  SAINT  model 
duplicates  the  RPV  flight  of  the  RPV/DCF  real-time 
simulation  as  discussed  in  Section  II.  In  performing 
this  duplication,  RPV  characteristics  and  the  RPV  simulation 
environmental  factors  are  input  to  the  SAINT  model  in  the 
same  form  and  manner  as  for  the  real-time  simulation.  This 
will  be  referred  to  as  " AMRL  data".  Thus,  the  parameter 
values  for  the  distributions  of  the  navigation  system  errors 
and  position  reporting  errors,  the  flight  plans  of  the  RPVs, 
the  parameters  for  jamming,  the  fuel  levels  on  the  RPVs, 
the  fuel  consumption  rates  of  the  RPVs,  the  malfunctions 
occurrring  to  the  RPVs,  and  the  type  of  each  RPV  are  input 
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to  the  SAINT  model  directly  from  AMRL  data.  The  state 
variable  portion  of  the  model  uses  these  values  to  duplicate 
the  RPV  flight  simulation  of  the  real-time  system.  The 
detailed  discussion  of  the  RPV  flight  simulation  as  designed 
in  SAINT  appears  in  the  technical  documentation  of  the  SAINT 
model  of  RPV/DCF  (9) . The  following  paragraphs  discuss  the 
SAINT  concepts  employed  in  the  simulation  of  RPV  flight. 

RPV  Flight  Positions 


As  in  the  real-time  simulation,  the  SAINT  model  keeps 
track  of  four  positions  for  each  RPV  at  all  times.  The 
first  position  is  the  true  position  (TP)  of  the  RPV.  This 
position  is  not  accessable  by  the  operators  of  the  system, 
but  must  be  maintained  to  monitor  the  actual  progress  of 
the  mission  for  statistical  purposes.  The  second  position,  the 
FPPE  or  flight  path  position  extrapolated,  represents  the 
position  that  the  RPV  would  be  at  if  it  was  following  the 
current  flight  path  perfectly.  Thus,  the  FPPE  is  the  true 
position  of  the  RPV  minus  any  accumulated  errors  in  flight 
that  the  RPV  has  made.  The  third  position  is  the  VFPPE  or 
virtual  flight  path  position  extrapolated.  It  is  the 
position  the  RPV  would  be  at  if  it  were  on  its  original 
flight  path  and  had  the  same  expected  time  of  arrival  at  its 
next  major  waypoint  based  on  the  current  command  speed  of 
the  RPV  (the  command  speed  of  the  RPV  is  the  speed  at 
which  it  is  supposed  to  be  flying) . Figure  11  is  a visual 
representation  of  these  RPV  positions  (the  fourth  position 
is  a position  report  (PR)  position  which  is  computed  at 
each  frame  update  from  the  actual  position  of  the  RPV) . 

The  RPV  was  originally  commanded  to  fly  from  point 
A through  B and  C to  the  major  waypoint  D.  However,  the  RPV 
flew  off  course  from  A to  B'  to  P.  At  P,  a directional 
patch  was  sent  to  the  RPV  instructing  it  to  fly  from  P 
through  X.  and  X„  to  D.  The  P-X. -X2  path  represents  the 
modified  rlight  path  or  the  flight  path  which  the  RPV  will 
attempt  to  fly.  Since  the  time  of  the  patch,  the  RPV  has 
again  flown  off  course  to  X'  and  TP.  At  this  point  in  time, 
the  FPPE  is  the  position  on^the  current  or  modified  flight 
path  where  the  RPV  should  be.  The  expected  time  of  arrival 
(ETA)  of  the  RPV  at  D is  the  distance  from  the  FPPE  to  X~ 
plus  the  distance  from  X~  to  D all  divided  by  the  command 
velocity.  The  VFPPE  is  the  point  on  the  original  flight 
path  where  the  RPV  would  be  based  on  this  ETA;  the  point 
on  the  original  flight  path  that  is  a distance  of  D to  X2 
plus  X2  to  FPPE  back  from  the  major  waypoint,  since  botli  the 
FPPE  and  VFPPE  are  based  on  the  same  velocity.  Both  the 
FPPE  and  PR  positions  are  used  by  the  operators  in  order 
to  control  the  mission.  The  TP  and  VFPPE  are  used  for 
statistical  evaluation  purposes  to  determine  the  actual 
deviation  from  the  flight  path  that  each  RPV  records. 


32 


Major  Waypoint  D 


VFPPE 


Actual  RPV  Flight 


Modified  Flight  Path 


Original  Flight  Path 


FPPE , and  VFPPE  for  an  RPV 


State  Equations  and  Modifications 


Six  state  variable  equations  are  employed  by  the 
SAINT  model  for  each  RPV  to  record  the  X and  the  Y coordinates 
for  the  TP,  FPPE,  and  VFPPE.  These  equations  are  of  the 
form  appearing  in  Figure  12  and  are  coded  in  FORTRAN  in 
subroutine  STATE,  shown  in  Figure  13.  They  are  automatically 
updated  by  SAINT  as  the  simulation  progresses.  The  SS (•  ) 
variables  used  by  SAINT  for  the  flight  path  positions  are 
defined  in  Table  V. 

The  values  of  the  variables  used  in  these  equations 
may  be  changed  by  operator  commands  or  onboard  computer 
instructions.  Whenever  a velocity  change,  at  the  request 
of  an  operator  in  the  SAINT  model,  is  processed,  the  v 
component  of  the  equations  of  Figure  12  is  changed.  For 
the  FPPE  and  VFPPE  equations,  this  v is  set  to  the  velocity 
requested  by  the  operator.  For  the  true  position  equations, 
the  v component  is  this  command  velocity  plus  the  ground 
speed  error  due  to  the  operating  navigation  system.  Similarly, 
when  an  operator's  patch  (request  for  directional  change)  is 
processed,  the  hx  and  h components  of  the  FPPE  equations  are 
changed  to  reflect  the  new  command  heading;  and  the  hx  and 
hy  components  of  the  true  position  equations  of  the  RPV  are 
set  to  the  FPPE  components  plus  the  ground  speed  error 
factors  resulting  from  the  operating  navigation  system.  In 
addition,  a new  ground  speed  error  is  determined  and  added 
to  the  command  velocity  to  obtain  a new  v value  for  the 
true  position  equations.  In  the  case  of  a patch,  the  VFPPE 
equations  may  or  may  not  change  heading  values  depending  on 
the  new  ETA  of  the  RPV  and  its  distance  to  its  next  major 
waypoint . 

In  response  to  the  original  RPV  flight  path,  the 
onboard  computer  may  also  cause  changes  in  the  position 
equations  of  the  RPVs.  The  flight  path  of  an  RPV  is 
described  by  a set  of  points  in  the  geographic  area  through 
which  the  RPV  is  to  fly  at  specific  velocities.  At  any 
point  in  time,  the  onboard  computer  of  the  RPV  carries 
instructions  for  all  future  maneuvers  the  RPV  is  to  perform. 
These  instructions  are  in  the  form  of:  "turn  to  a heading 
of  h beginning  at  time  t."  Thus,  the  points  on  the  flight 
path  are  transformed  into  turn  angles  and  turn  times  when 
they  are  received  by  the  onboard  computer.  This  transfor- 
mation is  performed  upon  flight  initialization,  as  well  as 
whenever  a velocity  change  or  directional  patch  is  sent 
to  the  RPV  by  the  operators. 

Time  is  the  controlling  factor  for  an  RPV  in  flight.  The 
RPV  always  assumes  that  it  is  on  course  and  will  make  heading 
changes  at  the  exact  time  prescribed  regardless  of  its  true 
position (errors  in  the  electronic  clocks  onboard  the  RPVs  are 
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is  the  current  value  of  simulated  time 


is  the  time  of  the  last  threshold  crossing  or 
task  completion 


is  the  x-coordinate  of  the  RPV  position  at 
time  t 


is  the  y-coordinate  of  the  RPV  position  at 
time  t 


is  the  heading  of  the  RPV  with  respect  to 
the  positive  x-axis 


is  the  cosine  of  the  heading 
is  the  sine  of  the  heading 
is  the  velocity  of  the  RPV 


Figure  12.  Equations  Governing  RPV  Positions. 
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SUBROUTINE  STATE 

***★★***★*★* 

COMMON  CARDS 

* A********** 

SS  (71) =SS ( 71 ) +DTNOW 

DO  10  1=1 , NRPV 

11=1+103 

12=1+119 

13=1+71 

14=1+87 

15=1+135 

16=1+151 

SS (II) =SSL(I1) +TSPED(I) *XHEAD (I) *DTNOW 
SS (12) =SSL (12) +TSPED(I) *YHEAD ( I ) *DTNOW 
SS (13)  =SSL(I3) +SPEED(I) *TXHED (I) *DTNOW 
SS (I4)=SSL(I4) +SPEED(I) *TYHED (I) *DTNOW 
SS ( I 5 ) =SSL (15) +TSPED ( I ) * VXHED ( I ) *DTNOW 
SS (16) =SSL(I6) +T3PED (I) *VYHED (I) *DTNOW 
10  CONTINUE 
RETURN 
END 


Figure  13.  Subroutine  STATE. 
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SAINT  SS(-)  VARIABLES  FOR  RPV  FLIGHT  POSITIONS 


SS ( • ) VARIABLES 

DEFINITIONS 

SS  (72) 

- SS ( 87 ) 

True  X-position  of  RPV 
I = 1-16 

SS  (88) 

- SS ( 103 ) 

True  Y-position  of  RPV 
I = 1-16 

SS  (104 ) 

- SS (119) 

FPPE  X-position  of  RPV 
I = 1-16 

SS (120) 

- SS (135) 

FPPE  Y-position  of  RPV 
1 = 1-16 

SS (136) 

- SS  (151) 

VFPPE  X-position  of  RPV 
I = 1-16 

SS  (152) 

- SS  ( 167 ) 

VFPPE  Y-position  of  RPV 
I = 1-16 

37 


1 


1 


considered  negligible).  Thus,  navigation  system  errors  will 
not  be  corrected  for  by  the  RPV  inflight.  The  RPV  assumes 
it  is  flying  exactly  as  it  should. 

The  RPVs  fly  from  coordinate  to  coordinate  in  a linear 
fashion.  The  simulation  of  RPV  flight  is  a linear  two- 
dimensional  approximation.  RPVs  fly  from  point  to  point  in 
straight  line  segments.  However,  this  approximation  does  not 
imply  that  RPV  maneuverability  is  ignored.  Rather,  all 
flight  paths  and  flight  path  modifications  are  initally 
computed  as  if  the  RPV  would  fly  a path  composed  of  linear 
segments  and  circular  arcs.  The  circular  arcs  are  then 
approximated  by  chords  that  never  exceed  fifteen  degrees 
of  turn. 

The  processing  of  flight  path  turns  in  the  SAINT  model 
is  accomplished  by  using  a combination  of  state  variables 
and  state  variable  monitors.  The  value  of  state  variable 
SS(I)  is  the  time  of  the  next  turn  on  the  FPPE  flight  path 
for  RPV  I.  The  value  of  SS  (71)  is  set  to  TNOW,  the  current 
value  of  simulated  time.  As  shown  in  Figure  14,  state 
variable  monitor  I is  used  to  monitor  the  values  of  SS(I)  and 
SS(71).  When  the  monitor  detects  the  value  of  SS(71)  crossing 
the  value  of  SS(I),  task  92  of  the  task-oriented  model  component 
is  signaled  and  the  value  of  logical  switch  1 is  set  to  I, 
the  number  of  the  RPV  preparing  to  turn.  Task  92  initiates 
a change  in  the  values  cf  hx  and  hy  in  the  FPPE  equations, 
and  the  values  of  hx,  hy,  and  v (including  navigation  system 
errors)  in  the  true  position  equations.  The  RPV  is  then  on 
course  to  its  next  flight  path  turn  point.  In  a similar 
fashion,  as  shown  in  Figure  14,  SS(36)  through  SS(51)  and 
monitors  17  through  32  signal  task  93  and  set  logical  switch 
2 for  processing  of  VFPPE  flight  path  turns. 

Monitor  33  is  used  to  detect  the  end  of  the  simulation. 

It  monitors  the  value  of  SS(168) , which  it  set  to  0 at  the 
start  of  the  simulation  and  2 when  all  RPVs  have  been  recovered. 
When  monitor  33  detects  the  value  of  SS(168)  crossing  a 
value  of  1,  task  94  is  signaled.  Task  94  is  a sink  task.  • Its 
completion  causes  the  end  of  the  simulation. 

Task-Oriented  Model  Component 

The  task-oriented  component  of  the  SAINT  model  of  the 
real-time  RPV/DCF  simulation  represents  the  control  and 
decision  tasks  performed  by  the  DCF  operators  during  a 
mission . 

The  majority  of  the  operator  tasks  in  the  SAINT  network 
are  either  tasks,  tasks  that  can  be  performed  by  any  of  the 
operators  specified.  The  use  of  either  tasks  allows  a single 
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SAINT  network  to  represent  the  control  of  all  RPVs  by  all 
DCF  operators.  In  actual  operation,  the  single  network 
represents  four  separate  networks  performed  by  four  distinct 
operators.  Information  and  operator  attributes  are  used 
extensively  to  provide  the  necessary  quantities  for  task 
performance  by  different  operators.  Operators  perform  all 
tasks  individually,,  in  an  order  dictated  by  the  task  pre- 
cedence relations. 

Information  attributes  flow  from  task  to  task  following 
the  precendence  relations.  In  the  SAINT  model,  they  are  used 
to  record  necessary  quantities  available  at  one  task  and  supply 
them  to  subsequent  tasks.  For  example,  operator  decision 
tasks  require  the  operator  to  search  through  his  list  of 
assigned  RPVs  to  determine  if  any  of  them  meet  a specified 
condition.  Whenever  an  RPV  is  found  to  meet  the  conditions 
prescribed,  an  information  attribute  is  assigned  the  RPV 
number.  In  addition,  a second  information  attribute  is 
assigned  a value  which  represents  the  condition  satisfied. 

The  quantities  assigned  to  information  attributes  provide 
necessary  information  for  subsequent  task  performance. 

In  the  real-time  simulation,  an  operator  makes  decisions 
as  to  when  and  how  to  perform  certain  tasks.  Some  of  these 
decisions  are  characteristic  of  the  particular  operator. 

Thus,  to  represent  the  system  accurately,  the  SAINT  network 
must  be  able  to  distinguish  between  operators.  Operator 
attributes  are  used  to  ensure  that  differences  in  operators 
are  represented.  Some  examples  of  the  operator  attributes 
used  in  the  SAINT  model  are:  1)  the  time  before  the  RPV 
reaches  its  handoff  coordinates  that  the  operator  prefers 
to  initiate  the  pop-up  maneuver;  2)  the  times  before  the 
RPV  reaches  its  handoff  coordinates  that  the  operator  prefers 
to  make  the  velocity  change,  the  altitude  change,  and  the 
handover  to  the  terminal  pilot  or  pseudo-pilot;  3)  the  lateral 
deviation  value  for  an  RPV  above  which  the  operator  will  make 
a directional  change  for  that  RPV;  and  4)  the  difference 
between  the  actual  ETA  and  the  desired  ETA  of  an  RPV  that  the 
operator  deems  acceptable. 

Another  point  to  be  made  here  concerns  task  performance 
times.  Some  tasks  representing  operator  actions  have  times 
that  are  primarily  operator  dependent.  For  example,  the 
time  that  it  takes  to  input  a velocity  change  is  the  time 
it  takes  the  operator  to  press  the  appropriate  buttons.  Actions 
such  as  this  have  times  that  are  governed  by  distributions. 

Other  tasks  in  the  network  are  needed  to  maintain  the 
structural  integrity  of  the  network  or  to  pass  information 

between  the  two  components  of  the  model.  These  tasks  take  j 

zero  time  to  perform.  There  are  some  tasks  whose  times  are 
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dependent  on  RPV  status  or  operator  attributes.  For  example, 
when  an  operator  wants  o make  an  RPV  directional  change,  he 
pushes  buttons  to  have  the  appropriate  information  displayed. 
However,  he  cannot  make  the  patch  until  the  information  is 
displayed.  Since  the  displays  appear  at  five-second  intervals 
he  may  wait  zero  to  five  seconds  for  this  update,  depending 
on  the  mission  time  at  which  he  made  his  request.  The  time 
for  this  task  is  computed  by  a moderator  function  as  the  time 
to  the  next  frame  update,  at  which  time  the  operator  inputs 
the  points  through  which  he  wants  the  RPV  to  fly.  Other 
examples  of  tasks  where  moderator  functions  are  used  to 
compute  performance  times  are  tasks  where  the  operator  waits 
for  the  results  of  his  most  recent  command;  tasks  where  the 
operator  waits  until  a specific  time  prior  to  the  RPV  reaching 
its  handoff  coordinates  before  he  performs  an  action;  and  the 
task  representing  the  operator  monitoring  mission  status 
whose  performance  time  depends  on  the  time  until  the  operator' 
next  scheduled  pop-up  maneuver.  Moderator  functions  are 
also  used  throughout  the  network  to  determine  or  alter 
task  performance  due  to  system  and  environmental  conditions 
and  to  record  operator  and  RPV  status  information. 

The  SAINT  network  is  shown  in  Figure  15.  The  basic 
structure  follows  the  form  of  operator  activities  as  outlined 
in  Figure  9.  A general  discussion  of  the  tasks  in  the  SAINT 
network  is  presented  below.  A complete  description  of  these 
tasks  appears  in  the  technical  documentation  (9) . 

Monitoring  Mission  Status 

The  SAINT  simulation  begins  with  all  enroute/return  ^ 
operators  observing  the  progress  of  their  assigned  RPVs  (18) x . 
This  task,  monitoring  mission  status,  is  performed  by  an 
operator  whenever  he  is  not  required  to  perform  any  other 
tasks  related  to  his  assigned  RPVs.  Following  the  monitoring 
of  mission  status,  operators  proceed  to  task  91,  where  they 
begin  a sequential  consideration  of  tasks  to  perform. 

Handover/Handback 

At  task  91,  the  operator  determines  if  it  is  time  to 
perform  a pop-up  maneuver  for  one  of  his  assigned  RPVs. 

If  so,  he  turns  on  the  flight  path  display  (S  RPVs),  waits 
until  his  preferred  time  to  input  the  altitude  change, 
and  performs  the  keyboard  functions  to  request  the  altitude 
change  (27).  He  then  waits  until  his  preferred  time  for 
the  velocity  change  and  performs  the  appropriate  keyboard 
functions  (29).  Following  the  velocity  change,  the  operator 


Numbers  in  parentheses  refer  to  the  tasks  numbers  as  shown 
in  Figure  15. 


Figure  15(2).  Task-Oriented  Component  of  the  SAINT  Simulation  Model 


waits  until  his  preferred  time  for  the  handover-prepare 
operation  and  inputs  the  prepare  command  (31).  If  this  is 
operator  1,  he  returns  to  task  91  to  determine  if  other  pop-ups 
need  to  be  performed.  Other  operators  wait  until  the  RPV  is 
handed  back  (34)  and  perform  the  pop-down  maneuver  (43). 

After  the  handover-prepare  command  has  been  input 
(31),  the  RPV  is  flown  through  the  target  area  by  the 
terminal  pilot  or  pseudo-pilot.  The  S RPVs  handed  to  the 
terminal  pilot  (operator  9)  are  accepted  by  the  pilot  (40) 
and  flown  through  the  target  area  (5).  RPVs  handed  to 
pseudo-pilots  are  accepted  (23)  and  flown  through  the  target 
area  (24).  RPVs  that  never  achieve  terminal  pilot  or  pseudo- 
pilot control  fly  through  the  target  area  according  to  their 
onboard  flight  paths. 

If  no  pop-up  maneuvers  are  called  for  at  this  time, 
the  operator  next  checks  to  see  if  the  terminal  pilot  or  a 
pseudo-pilot  has  released  an  RPV  to  him  (8).  If  so,  the 
activities  required  for  the  pop-down  maneuver  for  this 
RPV  are  determined  (43).  For  S RPVs,  the  velocity  is 
changed  and  the  flight  path  display  is  turned  off  (41) . 

For  all  RPVs  that  are  popped  down,  the  altitude  is  changed 
(42)  and  the  maneuver  is  completed  (47) . If  the  pop-down  was 
missed  (43),  the  maneuver  is  completed  (47)  without  the 
altitude  and  velocity  changes. 


If  operator  4 determines 
are  to  be  popped  down  at  this 
malfunction  has  occurred  (8). 
the  malfunction  (58),  reroute 
area  (68),  or  attempt  to  adju 
to  correspond  to  the  ETAs  of 
with  the  rerouted  RPV  (79) . 
or  if  this  is  not  operator  4, 
task  10. 


that  none  of  his  assigned  RPVs 
time,  he  determines  if  a 
If  so,  he  will  either  correct 
an  RPV  back  through  the  target 
st  the  ETA  of  a rerouted  RPV 
the  other  two  RPVs  associated 
If  no  malfunctions  have  occurred, 
processing  is  continued  at 


Enroute/Return 

If  no  pop-down  maneuvers  or  malfunction  corrections 
were  required  (8),  the  operator  determines  if  a velocity 
change  is  necessary  to  improve  the  ETA  of  one  of  his 
assigned  RPVs  (10) . If  so,  the  operator  computes  the 
new  velocity  and  inputs  the  velocity  change  command  for 
this  RPV  (48) . 


If  no  ETA  adjustments  are  required  at  this  time,  the 
operator  will  consider  patches  to  be  performed  (13) . Operators 
have  a variety  of  rules  to  determine  if  a patch  is  to  be 
performed.  Task  13  checks  the  rules  to  determine  if  a patch 
is  to  be  performed  and  directs  the  operator  to  the  appropriate 
task.  If  an  operator  decides  to  patch  an  RPV  because  its 


lateral  deviation  alarm  number  is  too  large,  he  requests  his 
preferred  patching  MOD  display  and  waits  for  the  frame  update 
(52).  He  then  performs  the  patch  (53).  If  an  operator 
checks  the  actual  lateral  deviation  before  making  a patch, 
he  requests  a MOD  display,  waits  for  the  frame  update,  and 
checks  the  lateral  deviation  value  (16) . If  the  value  is 
too  large,  he  patches  the  RPV  (53);  if  not,  he  checks  to  see 
if  any  other  of  his  assigned  RPVs  need  to  be  patched  (13) . 

If  an  operator  has  already  made  a patch  on  an  RPV  and  decides 
to  repatch,  he  performs  the  repatch  immediately  (53). 

Prior  to  performing  a patch  on  an  RPV,  the  operator 
determines  if  a turn  point  on  the  RPV  flight  path  is 
imminent  (53).  If  so,  he  delays  making  the  patch  (54) 
and  returns  to  consider  making  patches  on  other  RPVs  (13) . 

If  not,  the  MOD  display  preferred  by  the  operator  is 
accessed  (53),  and  the  operator  inputs  the  patch  and 
reconnect  points  (55).  After  inputting  a patch,  the 
operator  returns  to  consider  making  additional  patches  (13) . 


After  operator  4 completes  all  necessary  patches,  he 
determines  if  it  is  time  to  check  the  fuel  supply  of  each 
RPV  (13) . If  so,  the  time  of  the  fuel  check  is  recorded  (20) . 

Operator  4 will  then  determine  if  any  of  the  RPVs  are  in 
danger  of  not  being  able  to  reach  the  recovery  area  at 
their  present  fuel  consumption  rate  (21)  . If  an  RPV  is 
low  on  fuel,  he  changes  its  altitude  (56)  and  velocity  (57). 

Operator  4 then  continues  to  check  the  fuel  supplies  of  other 
RPVs  (21). 

After  all  necessary  patches  have  been  made  and  all 
fuel  levels  have  been  checked,  the  enroute/return  operators 
monitor  mission  status  (18) . 

Malfunctions 

Operator  4 corrects  malfunctions  (58)  whenever  he 
determines  that  one  has  occurred  (8) . If  the  malfunction 
has  already  been  corrected,  he  returns  to  other  duties  (8). 

If  the  malfunction  has  not  been  corrected,  he  determines 

the  type  of  malfunction  that  has  occurred  (58) . If  a 

navigation  system  malfunction  has  occurred,  operator  4 

determines  the  new  navigation  system  to  be  used  and  inputs 

the  navigation  system  change  command  (59) . For  a command 

link  malfunction,  he  drops  the  RPV  from  the  system  (62)  and  | < 

considers  a possible  rerouting  of  an  RPV  to  replace  the  one 

that  was  dropped  (64) . If  a terminal  malfunction  has 

occurred,  he  inputs  a destruct  or  chutes  command  (63)  and 

considers  a possible  reroute  (64).  If  an  RPV  was  lost  due 

to  enemy  or  terrain  consideration,  operator  4 proceeds 

directly  to  the  rerouting  determination  (64). 


Once  an  RPV  is  lost,  operator  4 determines  if  it  is 
possible  to  reroute  another  RPV  to  replace  it.  If  not,  he 
returns  to  his  other  duties  (91).  Otherwise,  rerouting 
indicators  are  set  (65)  and  the  velocity  of  the  two  RPVs 
associated  with  the  lost  RPV  are  decreased  (66,  67). 

If  a rerouting  operation  is  to  be  performed  (8) , operator 
4 determines  if  the  RPV  to  be  rerouted  has  been  handed-off 
to  the  terminal  pilot  or  a pseudo-pilot.  If  not,  he  resumes 
his  malfunctions  check  (8) . If  so,  he  increases  the  velocity 
of  the  RPV  to  be  rerouted  (69),  turns  on  its  flight  path 
display  (70)  , accesses  the  appropriate  MOD  display  and 
waits  for  the  frame  update  (71) . He  then  generates 
the  new  flight  path  (73)  and  waits  for  the  results  of  his 
rerouting  operation  (74) . 

Due  to  the  maneuverability  constraints  on  the  RPV,  it 
is  possible  for  the  rerouting  attempt  to  be  unsuccessful. 

If  it  is,  operator  4 will  try  again  (71).  He  will  attempt  to 
reroute  an  RPV  up  to  four  times.  Following  his  fourth 
unsuccessful  attempt,  he  makes  no  additional  attempts  and 
turns  off  the  flight  path  display  (80). 

If  operator  4 has  been  successful  in  generating  a 
flight  path  for  the  RPV  back  to  the  target  area,  he 
determines  if  the  ETAs  of  the  three  RPVs  involved  can  be 
synchronized  (76).  If  so,  the  flight  path  of  the  rerouted 
RPV  is  turned  off  and  the  new  ETA  requirements  are  determined 
(77).  If  the  ETAs  cannot  be  synchronized,  the  new  flight 
path  must  be  modified  (reprogrammed) . 

When  operator  4 determines  that  a rerouted  RPV  must  be 
reprogrammed  (8),  he  decides  if  an  attempt  should  be  made 
(79) . As  with  the  rerouting  operation,  operator  4 will 
make  only  4 attempts  to  coordinate  the  ETAs  of  the  three 
RPVs  involved.  In  addition,  he  will  discontinue  reprogramming 
attempts  if  one  of  the  two  slowed  RPVs  comes  within  two 
minutes  of  its  handover  waypoint.  If  operator  4 does  not 
attempt  a reprogram,  he  turns  off  the  flight  path  display  of 
the  rerouted  RPV  (80)  and  returns  to  other  duties. 

If  operator  4 decides  to  reprogram  an  RPV,  he  accesses 
the  MOD  1 display  and  waits  for  the  frame  update  (81) , 
inputs  the  new  flight  path  points  (83),  and  waits  for  the 
results  (84).  If  the  reprogram  was  successful,  he  will 
check  for  ETA  coordination.  If  not,  he  returns  to  his  other 
higher  priority  duties  (91)  before  attempting  another 
reprogram. 
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Interactions  Between  State  Variable  and  Task-Cr iented  Model 


Components 


From  the  SAINT  model  overview  provided  in  Figure  10,  it 
can  be  seen  that  two  distinct  mechanisms  are  required  for 
interaction  between  the  state  variable  and  task-oriented 
components  of  the  model.  First,  RPV  status  information  computed 
in  the  state  variable  component  must  be  made  available  to 
the  operators  in  the  task-oriented  component.  Second, 
commands  initiated  by  the  operators  must  be  supplied  to 
the  state  variable  component  for  processing.  Information 
attributes  and  user-written  support  routines  provide  the 
mechanism  for  the  necessary  interactions. 


RPV  Status  Information 


Task  95  represents  the  time  between  updates  of  the  CRT 
display  in  the  real-time  simulation.  This  task  is  completed 
every  five  seconds.  Upon  each  completion,  all  required 
RPV  status  information  is  updated.  This  information  includes 
the  status  of  the  communication  links  (up  or  down) , the  fuel 
remaining  onboard  the  RPVs,  the  estimated  (PR)  positions 
of  the  RPVs,  the  estimated  cross  track  errors  (lateral 
deviation)  of  the  RPVs,  and  the  lateral  deviation  alarm 
numbers  of  the  RPVs. 


Whenever  an  operator  requires  RPV  status  information 
in  order  to  make  a decision  at  a task,  the  user-written 
assignment  function  (function  USERF)  is  called.  This 
function  assigns  the  values  of  the  required  RPV  status 
variables  to  designated  information  attributes.  The  values 
of  these  information  attributes  are  then  used  as  branching 
parameters  in  the  network. 


As  an  example,  consider  the  decision  to  be  made  at 
task  16.  The  operator  must  determine  whether  or  not  to 
patch  an  RPV.  If  the  lateral  deviation  of  the  RPV  is 
greater  than  the  operator's  required  value,  information 
attribute  3 is  set  to  1.  This  causes  the  branch  to  task 
53  to  be  selected,  initiating  the  patching  process.  If 
the  lateral  deviation  is  not  greater  than  the  required 
value,  information  attribute  3 is  set  to  0,  allowing  the 
operator  to  return  to  task  13  to  consider  patches  on  other 
RPVs. 


In  the  manner  described  above,  all  RPV  status  information 
necessary  for  the  control  of  RPV  flight  by  the  enroute/return 
operators  is  passed  from  the  state  variable  component  of 
the  model  through  the  user-written  assignment  function  to 
information  attributes  for  use  by  the  operators  in  selecting 
tasks  to  perform. 
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Command  Processing 


Whenever  a 
the  SAINT  model 
command  be  sent 
assigned  values 
mation  attribut 
the  RPV  number 
which  specifies 
associated  with 
information  is 
portion  of  the 


task  in  the  task-oriented  component  of 
represents  the  operator  requesting  that  a 
to  an  RPV,  information  attributes  are 
that  completely  define  the  command.  Infor- 
ms 2,  4,  and  5 are  assigned,  respectively: 
t<  which  the  command  pertains;  a command  code 
the  type  of  command  requested;  and  the  value 
the  command  (if  one  is  required).  This 
then  directed  to  the  command  processing 
network,  beginning  with  task  86. 


Task  86  represents  the  time  delay  between  the  time  a 
command  is  requested  by  an  operator  and  the  time  it  is 
actually  received  by  the  RPV.  At  the  end  of  this  time, 
the  status  of  the  command  link  to  this  RPV  is  determined. 

If  the  command  link  is  down,  the  sending  of  the  command  will 
again  be  delayed,  after  which  the  command  link  status  check  is 
repeated  (96) . Whenever  the  command  link  is  found  to  be 
operative,  the  command  is  processed  (87).  Task  87  accesses  a 
user-written  moderator  function.  The  moderator  function 
passes  the  values  of  the  appropriate  information  attributes 
to  the  state  variable  component  where  the  processing  of  the 
command  is  completed. 

As  an  example,  consider  the  sequence  of  tasks  beginning 
with  task  57.  For  explanatory  purposes,  assume  that  RPV  15 
is  running  low  on  fuel.  This  situation  requires  operator  4 
to  decrease  the  velocity  of  RPV  15  to  310  knots.  To  make 
this  adjustment,  operator  4 must  perform  the  keyboard  functions 
required  to  input  the  command.  Task  57  represents  this 
operation. 

At  task  57,  the  information  attributes  are  assigned 
the  values  which  define  the  velocity  change  command. 

Information  attribute  4 is  assigned  a value  of  3,  the 
command  code  for  a velocity  change,  while  information 
attribute  5 is  assigned  a value  of  310,  the  desired  velocity 
(information  attribute  2 was  assigned  the  RPV  number  at 
task  22).  Upon  completion  of  task  57,  the  information  packet 
containing  the  attributes  described  above  is  directed  to 
task  86. 

At  task  86,  the  command  information  stored  in  the 
information  packet  is  delayed.  Following  the  delay, 
assuming  that  the  necessary  command  link  is  up,  the  packet 
moves  to  task  87  for  processing.  A moderator  function  is 
called  to  retrieve  the  attribute  values.  A user-written 
subprogram  is  called  from  the  moderator  function  with  the 
command  information  as  the  arguments.  The  subprogram  then 
alters  the  appropriate  state  variable  coefficients. 


SECTION  IV 


MODEL  EVALUATION  PROCEDURE  AND  PRESENTATION  OF  VALIDATION  RESULTS 


The  SAINT  model  of  the  RPV/DCF  real-time  simulation  was 
evaluated  by  comparing  the  values  of  the  system  performance 
measures  outputted  to  those  of  the  real-time  simulation. 

A single  sample  mission  performed  by  a single  set  of  operators 
was  selected  as  the  basis  for  the  evaluation.  The  mission 
selected  included  16  RPVs  and  was  performed  by  operator  group 
1.  Both  mission  profile  and  operator  characteristics  data 
serve  as  input  to  the  SAINT  model.  This  input  data  made  the 
general  SAINT  model  both  mission-specific  and  operator-specific. 
The  mission  profile  data  employed  was  identical  to  that  used 
by  the  real-time  simulation.  The  operator  characteristics 
used  represent  the  five  operators  that  comprise  group  1. 

The  RPV/DCF  real-time  simulation  was  designed  to  study 
system  performance  as  a function  of  system  parameters,  such 
as  mission  profile  and  RPV  characteristics.  For  this  purpose, 
the  real-time  simulation  outputs  statistical  measures  of 
system  performance.  The  SAINT  model  of  the  real-time  simulation 
was  designed  to  provide  the  same  or  expanded  outputs  of  system 
performance.  This  allowed  a direct  comparison  of  the  results 
of  the  two  simulations  to  be  made. 

Model  Evaluation  Procedure 


The  model  evaluation  procedure  employed  is  depicted  in 
Figure  16.  The  SAINT  simulation  was  run  and  the  detailed 
outputs  from  the  SAINT  and  the  real-time  simulations  were 
compared.  (Detailed  outputs  are  the  status  reports  for  the 
RPVs  presented  every  five  seconds.  Summary  statistics 
are  values  computed  after  a simulation  averaged  over  all 
RPVs.)  If  serious  discrepancies  between  correspoinding  outputs 
existed,  the  structure  of  the  SAINT  model  as  well  as  the 
parameters  of  the  model  were  studied  carefully  and  modifi- 
cations were  made  where  necessary.  The  simulation  was  then 
rerun  and  the  detailed  output  comparison  was  repeated. 

When  no  significant  differences  were  found  between  the 
detailed  outputs  of  the  SAINT  and  real-time  simulations,  the 
summary  statistics  generated  by  the  two  simulations  were 
compared.  Replications  of  the  SAINT  simulation  were  performed 
to  provide  a basis  for  statistical  analysis  and  validation. 

If  significant  differences  between  the  two  simulations  were 
discovered  at  this  point,  the  structure  and  parameters  of  the 
SAINT  model  were  re-analyzed  and  modifications  were  made 
where  appropriate.  After  these  alterations,  the  evaluation 
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Figure  16.  Model  Evaluation  Procedure 


process  continued  by  once  again  examining  the  detailed 
outputs  of  the  simulations. 
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The  entire  evaluation  procedure  was  continued  until 
no  significant  discrepancies  existed  between  either  the 
detailed  outputs  or  summary  statistics  of  the  two  simulations, 
or  in  other  words,  until  the  SAINT  model  was  found  to  be  a 
valid  representation  of  the  real-time  RPV/DCF  simulation 
for  the  mission  studied. 

During  the  model  evaluation  process,  two  major  types 
of  modeling  inaccuracies  were  discovered  that  resulted  in 
significant  discrepancies  between  the  SAINT  and  the  real- 
time outputs.  The  first  type  involved  inaccuracies  in  the 
modeling  of  operator  performance.  The  model  of  operator  duties, 
represented  by  the  task-oriented  component  of  the  SAINT 
model,  was  developed  through  actual  experience  on  the  system, 
observation  of  the  operators,  and  discussions  with  the 
operators.  However,  operators  do  not  always  function  as 
they  describe  and  are  not  always  consistent  in  their  actions. 
Thus,  in  some  cases  the  results  of  the  evaluation  procedure 
indicated  that  corrections  were  required  to  alleviate  the 
modeling  inaccuracies  that  resulted  from  the  misrepresentation 
of  operator  performance. 

The  second  major  type  of  modeling  inaccuracies  encountered 
resulted  from  the  misrepresentation  of  real-time  system 
performance.  In  some  cases,  parameter  values  used  in  the 
real-time  simulation  were  not  those  that  had  originally 
been  documented.  For  example,  the  ground  speed  navigation 
system  errors  for  the  RPVs  were  intended  to  have  means  that 
were  10  times  the  values  actually  used.  Thus,  when  the  SAINT 
model  used  the  values  specified  in  the  RPV/DCF  documentation, 
the  SAINT  outputs  were  found  to  be  significantly  different 
from  the  real-time  simulation  outputs.  In  another  case, 
the  flight  error  statistics  for  an  S RPV  between  its  S 
and  H waypoints  were  calculated  incorrectly  by  the  analysis 
routine  used  for  the  real-time  simulation.  Thus,  the  SAINT 
values  for  these  error  statistics  were  vastly  different 
from  the  real-time  values.  These  types  of  conditions  were 
corrected  by  altering  the  SAINT  model  design  to  reflect 
actual  real-time  system  performance  instead  of  documented 
system  design. 

Definition  of  Validity 

Ideally,  the  SAINT  model  should  be  considered  a valid 
representation  of  the  RPV/DCF  simulation  if  it  produces 
values  of  system  performance  measures  that  are  identically 
equal  to  those  of  the  real-time  simulation.  However,  since 
the  process  is  stochastic  in  nature,  the  ideal  definition 
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of  validity  is  not  applicable.  Under  the  stochastic  frame- 
work, the  model  should  be  considered  valid  if  the  distribution 
governing  individual  output  variables  is  statistically  the 
same  for  both  the  SAINT  and  real-time  simulations. 

Although  many  iterations  of  the  SAINT  model  can  be  run, 
an  additional  problem  was  encountered  when  trying  to  apply 
the  statistical  definition  of  validity  stated  above.  Only 
one  observation  of  the  real-time  simulation  output  variables 
was  available.  Thus,  no  estimate  of  the  parameters  governing 
the  distributions  of  the  real-time  simulation  output  variables 
could  be  obtained. 

A third  definition  of  validity  was  developed  which 
satisfies  both  the  constraints  stated  above  and  the  objectives 
of  this  effort.  This  definition  is: 

The  SAINT  model  is  considered  a valid 
representation  of  the  RPV/DCF  real- 
time simulation  if  the  observed  values 
of  the  system  performance  estimates 
generated  by  the  real-time  simulation 
could  have  been  obtained,  in  a 
probabilistic  manner,  from  the  dis- 
tributions governing  the  SAINT  system 
performance  estimates. 

t-test  Validation  Procedure 

The  validation  analysis  of  the  SAINT  model  was  performed 
using  two  separate  procedures,  depending  on  the  characteristics 
of  the  output  variable.  Cross  track  and  ground  speed  error 
are  sample  mean  statistics.  The  central  limit  theorem 
supplies  sufficient  justification  to  the  assumption  that  the 
underlying  probability  distribution  of  these  variables 
is  normal.  Thus,  the  validation  analysis  involves  a t-test 
of  the  real-time  value  with  respect  to  the  SAINT  distribution. 
For  purposes  of  validation,  25  replications  of  the  SAINT 
simulation  were  run  and  a 95%  confidence  two-sided  t-test 
was  performed  on  these  average  variables  to  test  the  feasibility 
of  obtaining  the  real-time  value  from  the  SAINT  distribution. 

Let  Xi  denote  the  value  of  the  SAINT  output  variable  for 
replication  i,i=2  ...  25.  Let  R denote  the  corresponding 
real-time  simulation  value.  Then: 

l 25 

X = SAINT  sample  mean  = ~ Z X.  ; 

25  i=l  1 

s = SAINT  sample  deviation 
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and  the  t statistic  is 

t = ~ R)  • 

S 

If  this  t statistic  is  greater  than  t ^5  = 2.06, 

the  hypothesis  that  the  real-time  simulation’ value  is  a 
representative  sample  from  the  SAINT  distribution  is  rejected. 
If  not,  the  hypothesis  is  accepted  and  the  model  is  assumed 
to  be  valid  for  the  variable  in  question. 


Some  clarification  of  the  acceptance  and  rejection  of 
the  hypothesis  should  be  made  at  this  point.  If  the  hypothesis 
is  rejected,  the  analysis  is  05%  certain  that  the  underlying 
distribution  of  the  SAINT  variable  could  not  produce  the  real- 
time value.  However,  there  is  a 5%  chance  of  rejecting  the 
hypothesis  when  it  is  true.  If  the  hypothesis  is  accepted, 
then  the  analysis  is  not  95%  certain  that  the  real-time  value 
is  not  representative  of  the  SAINT  distribution.  Thus, 
acceptance  does  not  supply  proof  that  the  hypothesis  is  true; 
it  only  indicates  that  the  hypothesis  cannot  be  disproved 
given  the  available  data.  However,  since  this  is  the  best 
validation  test  possible  with  only  one  observation  of  the 
real-time  system  output  variable  available,  the  acceptance 
of  the  hypothesis  for  an  output  variable  is  defined  as 
validating  the  model  for  that  variable. 

Histogram  Validation  Procedure 

The  second  procedure  performed  as  part  of  the  validation 
process  parallels  the  t-test  for  the  average  statistics. 

This  analysis  involves  variables  that  are  discrete  valued  and 
cannot  be  assumed  to  be  normal:  statistics  for  operator 
commands  for  individual  RPVs  and  command  processing  statistics, 
which  take  values  at  only  discrete  five  second  intervals  of 
time.  For  each  of  these  variables,  a histogram  was  drawn 
for  the  25  SAINT  replication  values  and  the  real-time  value 
was  indicated  on  the  histogram. 

The  SAINT  model  is  defined  as  being  valid  for 
these  variables  if  the  real-time  value  lies  within  the 
interval  whose  end  points  are  the  minimum  and  maximum  of 
the  SAINT  replication  values.  Thus,  the  model  is  defined 
as  being  valid  if  the  real-time  value  is  a feasible  output 
of  the  SAINT  simulation. 

Performance  Measures 

The  statistics  generated  by  the  SAINT  simulation  are 
constructed  to  provide  a detailed  comparison  of  the  outputs 
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from  the  SAINT  and  real-time  simulations.  The  statistics 
obtained  from  the  SAINT  model  duplicate  or  expand  on  the 
system  performance  statistics  generated  by  the  real-time 
simulation.  These  statistics,  to  be  defined  in  the  following 
paragraphs,  were  determined  to  be  representative  of  the 
system  performance  measures  generated  by  the  real-time 
system  and  provide  a rigorous  test  for  the  validity  of 
the  SAINT  model. 

RPV  Flight  Statistics 

The  statistics  recorded  by  the  SAINT  simulation  that 
concern  RPV  flight  and  are  averaged  over  RPVs  by  type 
(S,E,L)  are  identical  to  those  computed  by  the  real-time 
simulation.  These  statistics  are: 

1.  Average  cross  track  error  and  ground  speed 
error  during  the  enroute  phase  for  each  RPV  type 

2.  Average  cross  track  error  and  ground  speed 
error  during  the  return  phase  for  each  RPV  type 

3.  Average  cross  track  error  and  ground  speed 
error  between  S and  H for  S RPVs 

4.  Average  cross  track  error  and  ground  speed 
error  at  the  time  the  terminal  pilot  actually 
took  control  for  S RPVs 

5.  Average  cross  track  error  and  ground  speed 
error  at  H for  each  RPV  type 

6.  Average  cross  track  error  and  ground  speed 
error  at  S for  S RPVs 

7.  Proportion  of  attritions  for  each  RPV  type 

8.  Proportion  of  malfunctions  for  each  RPV  type 

9.  Proportion  of  recoveries  for  each  RPV  type 

The  above  statistical  categories  result  in  33  statistical 
variables  that  are  computed  by  averaging  the  appropriate 
statistic  over  the  RPVs  of  the  appropriate  type.  However, 
these  statistics  do  not  reflect  individual  operator 
differences  and  thus  are  not  sufficient  for  the  validation 
procedure.  SAINT  output  statistics  also  include: 

1.  Average  cross  track  error  and  ground  speed 
error  during  the  enroute  phase  for  each  RPV 


54 


1 


2.  Average  cross  track  error  and  ground  speed 
error  during  the  return  phase  for  each  RPV 

3.  Average  cross  track  error  and  ground  speed 
error  between  S and  H for  each  S RPV 

These  statistics,  coupled  with  those  listed  above,  provide 
a complete  comparison  between  the  SAINT  and  real-time 
simulations  in  terms  of  RPV  flight. 

Operator  Command  Statistics 

As  with  the  RPV  flight  statistics,  the  real-time  summary 
statistics  for  operator  commands  are  averaged  over  RPVs  by 
type.  These  include: 

1.  Number  of  patches  attempted  per  RPV  over 
all  mission  phases  for  each  RPV  type 

2.  Number  of  patches  successfully  completed 
per  RPV  over  all  mission  phases  for  each 
RPV  type 

3.  Number  of  velocity  commands  made  per  RPV 
over  all  mission  phases  for  each  RPV  type 

4.  Number  of  altitude  commands  made  per  RPV 
over  all  mission  phases  for  each  RPV  type 

5.  Number  of  patches  successfully  completed 
per  RPV  during  enroute  for  each  RPV  type 

6.  Number  of  patches  successfully  completed 
per  RPV  during  return  for  each  RPV  type 

7.  Number  of  reprogram  attempts  successfully 
completed  for  each  RPV  type 

These  statistics  are  computed  in  the  SAINT  simulation 
so  that  direct  comparisons  can  be  made.  However,  the 
statistics  do  not  reflect  operator  differences  and  thus 
are  not  sufficient  for  validation  purposes.  To  remedy 
this  situation,  the  SAINT  simulation  adds  the  following 
statistics  to  the  above  list: 


1.  Number  of  patches  attempted  during  enroute 
for  each  RPV 

2.  Number  of  patches  attempted  during  return 
for  each  RPV 


L J 
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3.  Number  of  patches  attempted  between  S and 
H for  each  S RPV 


Number  of  velocity  changes  during  enroute 
for  each  RPV 


Number  of  velocity  changes  during  return 
for  each  RPV 


6.  Number  of  patches  rejected  for  each  RPV 


These  statistics  provide  a complete  comparison  between 
the  SAINT  and  real-time  simulations  in  terms  of  number 
of  operator  commands  initiated. 


Command  Processing  Statxstics 


The  RPV/DCF  real-time  simulation  maintains  statistics 
on  the  proportion  of  RPVs  of  each  type  that  execute  the 
pop-up  maneuver  prior  to  H and  execute  the  pop-down  maneuver 
within  120  seconds  of  handback.  Since  these  statistics  do 
not  differentiate  between  RPVs  and  since  they  only  reflect 
gross  timing  averages,  they  were  found  to  be  inadequate  for 
validation  purposes.  The  SAINT  simulation  records  timing  values 
for  individual  RPVs.  These  values  are: 


The  time  of  processing  of  the  handover 
prepare  operation  for  each  RPV 


The  time  of  terminal  pilot  or  pseudo-pilot  handover 
accept  command  processing  for  each  RPV 


The  time  of  terminal  or  pseudo-pilot  handback 
command  processing  for  each  RPV 


The  time  of  processing  of  the  pop-down 
maneuver  for  each  RPV 


The  time  of  arrival  at  the  first  major  waypoint 
of  each  RPV 


These  statistics  provide  a complete  comparison  between  the 
SAINT  and  real-time  simulations  in  terms  of  handover- 
handback  timing  values. 


Analysis  of  Results 


For  each  statistic  type,  the  results  of  the  validation 
process  are  presented  in  tabular  form,  the  values  obtained 


summarized,  and  any  discrepancies  between  the  SAINT  and 
real-time  statistics  discussed.  The  discrepancies  arise  from 
the  fact  that  relatively  simple  moderator  functions  were  employed 
to  reflect  operator  performance.  The  discussion  of  these 
differences  as  they  relate  to  output  measures  focuses  on  the 
moderator  functions  that  require  improvement  to  more  accurately 
represent  actual  operator  performance.  The  development  of 
improved  moderator  functions  was  not  within  the  scope  of 
this  effort.  However,  the  results  obtained  were  more  than 
adequate  in  satisfying  the  project  objectives. 

RPV  Flight  Statistics 

The  t-test  analysis  presented  previously  has  been 
applied  to  the  RPV  flight  statistics  that  are  averaged 
over  RPVs  by  type.  The  results  of  this  analysis  are  given 
in  Table  VI.  For  each  variable,  the  table  presents 
the  sample  mean  and  standard  deviation  of  the  SAIN"11  output 
values,  the  95%  upper  and  lower  confidence  limits  for 
the  observations,  the  real-time  simulation  value,  the  t- 
statistic  for  this  test,  and  the  results  of  the  test  ("yes" 
for  accepting  the  model  as  being  valid  for  this  statistic 
or  "no"  for  rejecting  the  validity  of  the  model  for  this 
statistic).  From  this  table,  it  can  be  seen  that  the 
SAINT  model  has  been  determined  to  be  valid  for  30  of  the 
33  average  statistics.  The  three  statistics  that  are 
rejected  for  validity  are  the  cross  track  error  during 
enroute  for  E RPVs,  the  cross  track  error  during  return 
for  L RPVs,  and  the  ground  speed  error  during  return  for 
E RPVs. 

To  more  fully  define  these  three  statistics,  histograms 
of  the  SAINT  replicated  values  were  prepared,  with  the  real- 
time simulation  value  indicated.  These  histograms  appear 
in  Figures  17,  18  and  19,  respectively.  The  first  condition 
that  should  be  noticed  when  analyzing  these  histograms  is  that 
the  real-time  value  does  not  appear  to  be  significantly 
different  from  the  SAINT  values.  Thus,  the  rejection  of 
validity  for  these  variables  may  simply  be  due  to  the 
randomness  in  the  process.  However,  there  may  be  operator- 
oriented  reasons  behind  these  rejections.  The  SAINT  model 
assumes  the  operators  treat  E and  L RPVs  identically. 

Perhaps,  unconsciously  or  consciously,  the  operators  have 
a preferential  treatment  of  one  over  the  other  (it  is  known 
that  S RPVs  are  treated  differently  in  some  situations) . 

This  is  one  area  where  a deeper  analysis  of  human  characteristics 
might  improve  the  moderator  functions  included  in  the  model. 


2 

The  real-time  simulation  value  is  indicated  on  the  histograms 
presented  by  a 
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Figure  18.  Histogram  for  Cross  Track  Error  During  Return  for  L RPVs . 
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Figure  19.  Histogram  for  Ground  Speed  Error  During  Return  for  E RPVs. 


The  t-test  analysis  has  also  been  applied  to  the 
mission  performance  statistics  for  flight  path  error  of 
individual  RPVs . The  results  of  this  analysis  are  presented 
in  Table  VII.  These  results  indicate  that  the  SAINT  model 
is  a valid  representation  of  the  real-time  simulation  for 
69  of  the  74  variables.  The  five  rejected  are  the  ground 
speed  error  during  enroute  for  RPV  5,  the  ground  speed 
and  cross  track  errors  during  return  for  RPV  7,  and  the  cross 
track  and  ground  speed  errors  during  return  for  RPV  14. 

To  more  fully  explain  the  rejections,  histograms 
of  the  SAINT  values  of  the  rejected  variables  were  generated 
with  the  real-time  value  indicated.  These  histograms  appear 
in  Figures  20,  21,  22,  23,  and  24,  respectively.  The  ground 
speed  error  during  enroute  for  RPV  5 may  be  higher  in  the 
real-time  simulation  due  to  randomness  or  due  to  some  differ- 
entiation of  this  RPV  from  others  by  the  controlling  operator. 
The  other  four  rejected  statistics  seem  to  display  some 
consistency.  The  SAINT  operator  provides  lower  cross  track 
and  ground  speed  errors  for  RPV  7 on  return.  This  seems 
to  point  to  the  fact  that  operator  4,  who  is  modeled  as 
treating  all  RPVs  identically  on  return,  actually  has  less 
concern  with  his  S RPV  during  the  return  phase.  Also,  SAINT 
operator  2 seems  to  direct  RPV  14  for  return  better  than 
the  real-time  operator.  Operators  3 and  4 are  modeled  as 
treating  their  extra  RpVs  differently  on  return;  perhaps 
so  should  operator  2.  These  results  indicate  additional 
areas  that  should  be  analyzed  in  the  development  of  moderator 
functions. 

Operator  Command  Statistics 


Operator  command  statistics  are  of  two  types:  those 
averaged  over  RPVs  by  type  and  those  for  individual  RPVs. 

The  t-test  analysis  has  been  applied  to  those  operator  command 
statistics  averaged  over  RPVs  by  type.  The  results  of  this 
analysis  appear  in  Table  VIII.  As  these  results  indicate, 
the  SAINT  model  is  a valid  representation  of  the  real-time 
simulation  for  all  18  variables  analyzed. 

The  operator  command  statistics  for  individual  RPVs 
are  tested  for  validation  based  on  the  real-time  value  in 
relation  to  the  histogram  of  the  SAINT  replication  values. 

For  each  variable,  the  following  test  is  made:  If  the 

real-time  value  lies  within  the  minimum  and  maximum  of  the 
SATNT  values,  the  model  is  assumed  valid.  The  results 
of  this  analysis  are  presented  in  Table  IX.  The  histograms 
for  selected  operator  command  statistics  appear  in  Appendix  I. 
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Table  VII  continued 
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Figure  20.  Histogram  for  Ground  Speed  Error  During  Enroute  for  RPV 
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Figure  21.  Histogram  for  Cross  Track  Error  During  Return  for  RPV 


Figure  22.  Histogram  for  Ground  Speed  Error  During  Return  for  RPV 
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Figure  23.  Histogram  for  Cross  Track  Error  During  Return  for  RPV  14 


cO 


a o o o o 
CD  O vX>  -O  'T' 
O O »»»•*)  C\i 


a o 


CD  CD  CD  O CD 
C*  CD  CD  CD  CD 
CD  CD  C O CD 


oo  a oo 


n o o o o 
cocao 
o o a o a 


3 0 0 0 0 


CD  CD  O O CD 
CD  CD  O CD  CD 
O O O O O 


o o o o o 


o o c o a 
n o o c o 
o o o o o 


o o o o o 


O O U u)  O 

cd  o o cd  a 

O CD  C O CD 


CD  CD  o O O 


OOa'a'r^OCOCDOCCDOOOOOCDOOOOOOOOOOOCD 


OC3CDCDCDCDcaaOCDCDOCDOCDCDCDCDCDC*OrDOCDOOCr>CDOCD 

in  c ii\  o ut  o u,  o in  o ir>  o a>  c ir>  o ut  o o in  c in  o ui  □ it  o tn  o 


t rioiftvaiONN.ffiro'O'ooH^fVjivjMMj  s in 


I • I I I I I • I • I I I I I I t • I I I I I I I I I I I I 

O CD  O O O O CD  O O CD  CD  O CD  O O O CD  O O CD  O CD  O CD  C CD  O O CD  O 

u>  o ui  o O'  o u,  o a\  o ir,  o ifi  o ai  o u,  o ui  o ci  o ^ c un  o u,  o u, 

HHi\ir\jroMD4Uiiiiifla*ssco(c(7'C''oo»-i»-(Mtvj(*5ff;j 


Figure  24.  Histogram  for  Ground  Speed  Error  Duriug  Return  for  RPV  14. 
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Discrepancies  for  command  statistics  occur  in  only 
two  of  the  85  variables:  the  number  of  patches  on  return 
for  RPV  13  and  the  number  of  velocity  changes  on  enroute 
for  RPV  15.  The  histograms  for  these  statistics  appear 
in  Figures  25  and  26.  These  differences  could  be  due 
to  randomness,  to  the  patching  criteria  employed,  or  to 
the  manner  in  which  the  operators  determine  the  new  velocity 
for  an  ETA  prompted  velocity  change.  These  areas  require 
further  analysis  as  they  relate  to  moderator  functions. 


Command  Processing  Statistics 

A histogram  analysis  was  performed  for  the  command 
processing  statistics  collected.  The  results  of  this 
analysis  appear  in  Table  X.  Histograms  for  selected 
command  processing  statistics  appear  in  Appendix  II. 
These  results  show  that  in  all  but  6 of  the  71  cases, 
the  model  is  valid.  The  exceptions  are: 
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The  histograms  for  these  statistics  are  shown  in  Figures 
27  through  32.  The  histograms  indicate  that  RPV  10  is 
handed-back  to  the  enroute  recovery  operator  and  popped- 
down  sooner  by  the  real-time  operators  than  the  SAINT 
operators.  Although  the  deviations  between  the  two  are  not 
large,  they  do  point  to  the  need  for  an  analysis  of  terminal 
flight  by  the  pilots,  i.e.,  when  and  why  does  the  pilot 
release  control  of  the  RPV.  This  could  be  random  condition 
or  one  that  has  a relationship,  for  example,  to  lateral 
deviation  of  the  RPV  at  the  time  of  terminal  pilot  control. 

RPV  12  and  RPV  15  are  both  popped-down  later  by  the  real- 
time operators  than  by  the  SAINT  operators.  This  indicates 
that  the  perception  of  a required  pop-down  may  not  be  as 
immediate  as  assumed  in  the  SAINT  model.  Also,  the  handover- 
prepare  for  RPV  12  and  handover-accept  for  RPV  15  are  actually 
performed  later  than  predicted  by  the  SAINT  model.  Thus, 
while  the  time  values  employed  by  the  SAINT  operators  to 
determine  the  time  of  operations  generally  produce  timing 
statistics  that  agree  with  the  real-time  values,  there  are 


Figure  26.  Histogram  for  Number  of  Velocity  Changes  During  Enroute  for  RPV  15. 
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Figure  27.  Histogram  for  Time  of  Handback  for  RPV  10. 
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Figure  28.  Histogram  for  Time  of  Pop-down  for  RPV  10. 


Figure  29.  Histogram  for  Time  of  Handover-Prepare  for  RPV  12 
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Figure  30.  Histogram  for  Time  of  Pop-down  for  RPV  12 
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Figure  31.  Histogram  for  Time  of  Handover  Accept  for  RPV  15. 


Figure  32.  Histogram  for  Time  of  Pop-down  for  RPV  15 


SECTION  V 


CONCLUSIONS 


The  results  of  the  evaluation  of  the  SAINT  model  of 
the  RPV/DCF  simulation  show  that  the  SAINT  model  is  a 
satisfactory  representation  of  the  one  team-one  mission 
real-time  simulation  for  265  of  the  281  measures  of  system 
performance  analyzed,  or  94%.  These  results  are  sufficiently 
conclusive  in  satisfying  the  objectives  of  this  effort. 

In  addition,  the  SAINT  simulation  model  proved  to  be  more 
efficient,  in  terms  of  computer  processing  time,  than  the 
real-time  simulation.  The  SAINT  simulation  of  the  one 
team-one  mission  scenario  required  approximately  five  minutes 
of  computer  processing  time,  while  the  real-time  simulation 
of  the  same  scenario  required  approximately  ninety  minutes. 

In  addition  to  providing  a satisfactory  representation 
of  the  real-time  simulation,  the  SAINT  model  proved  to  be 
invaluable  in  locating  inaccuracies  in  the  real-time 
simulation  program.  This  fact  indicates  the  usefulness  of 
a SAINT  model  as  an  independent  back-up  simulation  of  a 
real-time  effort.  Using  SAINT  as  in  this  effort,  a dual 
model  of  a real-time  simulation  can  be  constructed  and 
exercised.  The  outputs  of  the  two  systems  can  be  compared. 
This  analysis  will  indicate  inaccuracies  in  the  real-time 
simulation  as  well  as  inaccuracies  in  the  SAINT  model. 

This  dual  simulation  approach  offers  substantial  savings  in 
time  and  output  loss  due  to  inaccuracies  in  the  real-time 
simulation  and  a substantial  increase  in  the  confidence  of 
the  results  of  both  simulations. 

For  the  majority  of  statistics  analyzed,  the  results 
indicate  that  the  operator  performance  moderator  functions 
developed  and  employed  in  the  SAINT  model  are  satisfactory. 
However,  the  results  do  point  to  a need  for  additional 
moderator  function  development,  and  indicate  that  ouch 
development  will  provide  improved  results. 

The  SAINT  model  of  the  RPV/DCF  real-time  simulation 
has  been  successfully  developed.  It  has  been  satisfactorily 
evaluated  with  respect  to  one  mission  of  the  real-time 
simulation.  The  applicability  of  the  SAINT  modeling  technique 
to  the  analysis  of  the  RPV/DCF  system  has  been  demonstrated. 


Since  the  real-time  simulation  includes  complex  man- 
machine  interactions,  the  usefulness  and' power  of  the  SAINT 
technique  in  representing  those  interactions  have  been 
demonstrably  shown.  A complex  man-machine  system  can  be 
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modeled  and  simulated  using  SAINT.  Thus,  the  system  can  be 
studied  in  a controlled  environment  without  disruption 
to  its  operation  and  at  a relatively  low  cost.  Certainly, 
expertise  in  the  area  of  human  performance  is  required  to 
provide  generally  applicable  moderator  functions  that 
predict  operator  efficiency  and  operator  decisions  based 
on  system  characteristics.  However,  SAINT  provides  the 
framework  through  which  this  expertise  can  be  effectively 
channeled,  and  through  which  system  performance  measures 
can  be  obtained  based  on  subsystem  descriptions. 


SECTION  VI 


RECOMMENDATIONS 


The  SAINT  model  of  the  RPV/DCF  simulation  has  been 
shown  to  be  a satisfactory  representation  of  the  one  team, 
one  mission  scenario  that  was  analyzed.  While  this  is  a 
very  significant  result,  the  model  should  be  evaluated  for 
additional  missions  and  operator  groups.  When  this  is 
completed,  the  SAINT  model,  in  conjunction  with  the  RPV/DCF 
real-time  simulation,  will  be  an  extremely  useful  predictive 
and  analytical  tool  for  the  evaluation  of  RPV/DCF  system 
design  and  for  the  analysis  of  operator  performance  in 
relation  to  varying  system  parameters.  This  section  outlines 
the  procedure  necessary  to  obtain  such  a model. 


The  real-time  RPV/DCF  mission  analyzed  in  this  effort, 
as  shown  in  Section  IV,  includes  16  RPVs  and  is  performed 
by  operator  group  1.  The  next  stage  of  the  development 
effort  should  be  to  obtain  a SAINT  model  that  satisfactorily 
represents  all  16  RPV  missions  performed  by  operator  group  1. 
In  order  to  achieve  such  a model,  it  is  anticipated  that  the 
human  performance  moderator  functions  included  in  the 
original  model  will  need  further  development  and  refinement 
in  the  following  areas: 


1.  Preferential  treatment  of  one  type  of  RPV  over 
another  by  an  operator 


2.  Preferential  treatment  of  individual  RPVs  over 
others  by  an  operator 


Determination  of  a new  velocity  for  an  RPV  for 
purposes  of  ETA  manipulation  by  an  operator 


The  time  of  acceptance  and  release  of  control  of 
an  RPV  by  the  terminal  pilot 


The  time  of  handover  and  handback  operations  for 
an  RPV  by  an  operator 


Unsuccessful  or  unattempted  performance  of  a required 
task  by  an  operator 


In  addition  to  the  refinement  of  moderator  functions,  some 
alterations  to  the  structure  and  parameters  of  the  model 
may  be  necessary.  It  should  be  noted  that  the  evaluation 
procedure  employed  at  this  point  will  be  more  rigorous  than 
that  of  Section  IV  due  to  the  increase  in  the  number  of 
available  observations  of  the  real-time  simulation. 


*>  1**- 


Once  a representative  model  for  all  16  RPV  missions 
performed  by  operator  group  1 has  been  developed,  the  model 
should  be  evaluated  for  missions  performed  by  the  same 
operator  group  that  include  an  increased  number  of  RPVs. 

Once  again,  the  human  performance  moderator  functions  developed 
in  the  previous  step  may  require  additional  refinement.  In 
addition,  this  stage  of  the  development  process  will  provi.de 
a check  on  the  task  performance  times  specified  in  the 
original  model.  In  that  model,  it  was  assumed  that  task 
performance  times  were  dominated  by  frame  update  times. 

If  the  assumption  proves  to  be  invalid  when  applied  to  missions 
consisting  of  more  than  16  RPVs,  appropriate  adjustments 
need  to  be  made  to  the  task  performance  times  and/or 
associated  moderator  functions. 

Once  the  model  has  been  satisfactorily  evaluated  for 
all  missions  performed  by  operator  group  1,  the  analysis 
shifts  to  other  operator  groups.  For  each  group  of  operators 
selected,  the  procedure  will  be  similar  to  that  used  for 
operator  group  1.  The  objective  is  to  achieve  a represen- 
tative model  for  all  operator  groups  and  all  missions. 

When  the  SAINT  model  of  the  RPV/DCF  simulation  is 
satisfactorily  evaluated  for  all  operator  groups  and  all 
missions,  it  will  be  an  extremely  useful  and  low  cost 
tool  that  can  be  used  in  conjunction  with  'the  real-time 
simulation  to  analyze  RPV/DCF  design.  It  can  be  used  for 
feasibility  studies  and  value  analyses  of  system  design 
alternatives  prior  to  their  implementation  in  the  real-time 
system,  as  well  as  providing  an  additional  verification 
of  the  operation  of  the  real-time  simulation  program. 
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APPENDIX  I 


HISTOGRAMS  FOR  OPERATOR  COMMAND  STATISTICS 


This  appendix  presents  the  histograms  for  selected 
operator  command  statistics  used  for  the  validation 
analysis  presented  in  Section  IV.  The  histograms  that 
appear  in  this  appendix  are  listed  below  in  the  order 
in  which  they  appear: 

1)  Number  of  patches  during  enroute  for  each 
of  the  16  RPVs 

2)  Number  of  patches  during  return  for  each 
of  the  1 6 RPVs 

3)  Number  of  velocity  changes  during  enroute 
for  each  of  the  16  RPVs 

4)  Number  of  velocity  changes  during  return  for 
each  of  the  16  RPVs 

The  real-time  values  for  these  variables  are  indicated  by 
a tt  on  these  histograms. 
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This  appendix  presents  the  histograms  for  selected 
command  processing  statistics  used  for  the  validation 
analysis  presented  in  Section  IV.  The  histograms  that 
appear  in  this  appendix  are  listed  below  in  the  order 
in  which  they  appear: 


1)  Time  of  the  pop-up  maneuver  for  each  of  the 
16  RPVs 


Time  of  the  pop-down  maneuver  for  each  of  the 
15  RPVs  that  were  popped-down  (RPV  3 was  not 
popped-down) . 
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